What's new in MAAS (snap/2.7/cli)

2.7 2.8
Snap CLI ~ UI CLI ~ UI
Packages CLI ~ UI CLI ~ UI

MAAS 2.7 release notes

MAAS 2.7.3 released

On 24 August 2020, MAAS 2.7.3 was released, replacing the 2.7/stable channel in snap and the ppa:maas/2.7. You can update your 2.7 release to 2.7.3 by with:

snap refresh --channel=2.7/stable

or by using the aforementioned PPA. The focus for this release has been bug-fixing – there were no changes to MAAS since RC1.

Thanks to everyone who reported the issues with previous 2.7 releases and helped us with the logs.

MAAS 2.7.2 released

On 30 July 2020, MAAS 2.7.2 was released, replacing the 2.7/stable channel in snap and the ppa:maas/2.7. You can update your 2.7 release to 2.7.2 by with:

snap refresh --channel=2.7/stable

or by using the aforementioned PPA. The focus for this release has been bug-fixing – there were no changes to MAAS since RC1.

Thanks to everyone who reported the issues with previous 2.7 releases and helped us with the logs.

Upgrading from MAAS 2.6 snap

If you are using the MAAS 2.6 snap, which had to be installed with --devmode, you can update to 2.7 with the following parameters:

snap refresh maas --devmode --channel=2.7

Be aware that you will still be in --devmode, which means the snap won’t automatically upgrade. You’ll have to check manually for updates (via snap refresh). Once you’re upgraded to MAAS 2.7 using this method, future snap updates won’t require the devmode parameter. So, for example, when a later version of 2.7 (or even 2.8) is released, you will be able to snap refresh to those channels and get out of devmode. Once refreshed out of devmode in this way, you’ll get updates for point releases automatically.

An alternative to avoid devmode would be to do a clean install of MAAS 2.7, that is, removing 2.6 with snap remove maas and reinstalling MAAS 2.7 with:

snap install --channel=2.7 maas

Note that you can check the devmode status of your snap with:

snap list maas

MAAS 2.7 released

Following on from MAAS 2.6.2, we are happy to announce that MAAS 2.7 is now available. This release features some critical bug fixes, along with some exciting new features:

CentOS 8 image support

Users can now deploy CentOS 8 images in MAAS. The Images page in the MAAS UI will now offer a choice to select and download CentOS 8. Note that users of previous versions may see CentOS 8 as an available option, but cannot download or deploy it.

Network testing features

MAAS 2.7 brings a slate of new network testing and link detection features, as detailed below.

Network link disconnect detection

MAAS 2.7 can check whether links are connected or disconnected. Previously, when commissioning, you couldn’t detect unplugged cables. Now you can. You will have to take a couple of steps for existing MAAS deployments: First, you will have to upgrade to 2.7, then run commissioning again to see if a link is disconnected. But you no longer have to puzzle over what’s broken when this happens.

MAAS will report disconnected network cables, and users will receive a warning when trying to configure a disconnected interface. Administrators will be able to change cable connection status through both API and UI after manually rectifying the situation.

Slow network link detection

MAAS 2.7 makes sure you’re getting the most out of your link speed. As servers and hardware get faster — 10G, 40G, even 100G NICS — the chances increase that you might plug your 10G NIC into a 1G switch, for example. Previously, with MAAS, you’d be stuck with the speed of the slowest link, but there wasn’t a way to verify your link speed without recommissioning. Depending on your physical hardware, that might still be an issue, but the MAAS UI can now warn you if your interface is connected to a link slower than what the interface supports. And all information shown in the UI is available via the API, as well. You can still replace a slow switch without recommissioning.

MAAS will automatically detect link and interface speed during commissioning and report them via the API/UI. Administrators will be able to change or update the link and interface speeds via the API/UI after manual changes to the connection. MAAS 2.7 will also report link speed, allowing users to filter and list machines by their link speed in the UI. Users can also employ this information to allocate machines by their link speed in the API.

Network validation scripts and testing

MAAS 2.7 allows you to configure network connectivity testing in a number of ways. If you’ve used MAAS, you know that if it can’t connect to the rack controller, deployment can’t complete. Now MAAS can check connectivity to the rack controller and warn you if there’s no link, long before you have to try and debug it. For example, if you can’t connect to your gateway controller, traffic can’t leave your network. MAAS can now check this link and recognise that there’s no connectivity, which alleviates a lot of annoying (and sometimes hard-to-detect) network issues.

Users can now test their network configuration to check for:

  1. Interfaces which have a broken network configuration
  2. Bonds that are not fully operational
  3. Broken gateways, rack controllers, and Internet links

In addition, Internet connectivity testing has been greatly expanded. Previously, MAAS gave a yes/no link check during network testing. Now you can give a list of URLs or IP addresses to check. In the ephemeral environment, standard DHCP is still applied, but when network testing runs, we can apply your specific configuration for the duration of the test. While all URLs / IPs are tested with all interfaces, we do test each of your interfaces individually, including breaking apart bonded NICS and testing each side of your redundant interfaces. You can also run different tests on each pass, e.g., a different set of URLs, although each run would be a different testing cycle. For testing individual interfaces, you can use the API.

Of course, the main network feature available in 2.7 is improved — and customisable — network testing. You can now create your own commissioning scripts and tests related to networking. You can create your own network tests (e.g., a network throughput test) and run them during the network testing portion of the MAAS workflow. There are no particular restrictions on these scripts, so you can test a wide variety of possible conditions and situations.

Administrators can upload network tests and test scripts, as well as create tests which accept an interface parameter, or scripts which apply custom network configurations. Users can then specify (unique) parameters using the API, override machines which fail network testing (allowing their use), and suppress individual failed network tests. All users benefit from enhanced reporting, as they are now able to see the overall status of all interfaces via the API, the UI Machine list, and the UI Interfaces tab; review the health status from all interface tests; and view the interface test results by interface name and MAC.

Live IP address detection to prevent address conflicts

In some cases, MAAS connects with subnet which are not empty. Normally, the user has to tell MAAS about IP addresses which are already assigned on that subnet, and if that step is skipped, MAAS may assign and in-use IP address to one of the machines under its control, leading to an IP conflict.

MAAS 2.7 alleviates this problem by detecting IPs in use on a subnet, so that it can avoid assigning that IP to a MAAS-managed machine. The system is not perfect; for example, if NIC on a subnet-connected machine is in a quiescent state – or turned off – MAAS may not detect it before duplicating the IP. Note that at least one rack controller must have access to the previously-assigned machine in order for this feature to work. MAAS 2.7 will also recognise when the subnet ARP cache is full and re-check the oldest IPs added to the cache to search for free IP addresses.

Introductory NUMA / SR-IOV support

NUMA (Non-Uniform Memory Access) is a useful way of achieving high-efficiency computing, by pairing a CPU core with a very fast connection to RAM and PCI buses. Typically the CPU socket and the closest banks of DIMM constitute a NUMA node. Obviously, if you’re deploying a MAAS machine under NUMA to get maximum performance, you would like for that machine to be confined to a single NUMA node. MAAS 2.7 introduces this capability.

MAAS will display the NUMA node index and details. Users can also see the count of available NUMA nodes, along with CPU cores, memory, NICS, and node spans for bonds and block devices (although node-spanning may not produce suitable performance). From a reporting standpoint, users can filter machines by CPU cores, memory, subnet, VLAN, fabric, space, storage, and RAID.

Similarly, the SR-IOV (Single Root I/O Virtualisation) allows a PCIe device (e.g, a NIC) to appear to be multiple separate devices. A network adaptor can be subdivided into multiple adaptors by adding a Virtual Function (VF). MAAS 2.7 supports the use of multiple VF adaptors to intelligently use SR-IOV edge clouds, by allowing users to see that a NIC supports SR-IOV, along with the supported VF counts.

The goal of this feature is to help users choose the right machine to deploy an edge cloud.

Settings and user preferences redesign

As part of our efforts to make the UI faster and more responsive, we have completely redesigned the Settings and User preferences within the MAAS UI.

Strictly-confined Snap support

With 2.7, MAAS is fully confined within the Snap container. No need for installation qualifiers (such as “–devmode”) to permit use of external resources, i.e., outside the Snap container. This means that we will begin to transition to recommending the Snap install as the default (and primary) MAAS install method. This also means that MAAS now gains the benefit of confined snap security features.

Update to hardware information gathering methods

MAAS has switched hardware information gathering from lshw/lsblk to lxd output during commissioning, because it more easily provides the information needed to complete resource discovery. Note that this information may not be particularly reliable for your use, so you may need to create your own commissioning scripts if you need something more detailed or specific.

Bug fixes

A number of bug fixes (see the list in Launchpad).

Cumulative summary of new features in MAAS 3.0

  1. PCI and USB devices are now modelled in MAAS
  2. IBM Z DPM partition support
  3. Proxmox support
  4. LXD projects support
  5. PCI and USB device tabs in machine details
  6. Workload annotations
  7. Fixed status bar
  8. Registering a machine as a VM host during deployment
  9. Improvements to MAAS CLI help UX
  10. Disabling boot methods
  11. Consolidation of logs and events

MAAS 3.0 bug fixes

MAAS 3.0 incorporates a large number of bug fixes, summarised in the sections below. Please feel free to validate these fixes at your convenience and give us feedback if anything doesn’t seem to work as presented in the bug request.

One particular bug, #1916860, involves failures in the IPMI cipher suite in MAAS 2.9.2 and up, on the Lenovo x3650 M5 (and others). This particular bug is a not a MAAS bug, but a firmware issue with the subject machines. While the MAAS team can’t fix this (hence the assignment of “Won’t Fix”), the team did provide a easy workaround which helps circumvent this issue.

MAAS 3.0 bug fixes

Here are the bugs that were ‘Fix Released’ for the MAAS 3.0 release:

Number Description Importance
#1932136 interface with a warning is not configured properly Critical
#1896771 interfaces that are not connected are detected as ‘connected to slow interface’ Medium

MAAS 3.0 RC2 bug fixes

Here are the bugs that have been ‘Fix Released’ in MAAS 3.0 RC2:

Number Description Importance
#1929552 Deb-based controller fails to run machine-resources Critical
#1929576 Machines fail to commission using the 3.0 snap due to possible? DNS issue Critical
#1930227 Failure to commission when interfaces has a /32 IP Critical
#1930554 vm-host CLI command is now named vmhosts Critical
#1930587 Different disks with same LUN detected as multipath Critical
#1931215 [.0~rc2-10023 testing] two IPs assigned to one interface Critical
#1931838 Reverse DNS lookup fails for subnets smaller than /24 Critical
#1835292 UI should add button to download curtin-logs.tar on deployment failure MAAS High
#1908552 maas init fails; ‘relation “maasserver_routable_pairs” does not exist’ High
#1929086 LXD VM hosts can’t be refreshed if VLANs interfaces aren’t named $parent.$vid High
#1929643 MAAS often fails and and returns a Pickled object if request header is set to Accept: / Medium
#1924820 Trying to edit a disconnected NIC, then cancelling the edit and connecting the NIC via its drop-down menu, many drop-down menu options then disappear Undecided

MAAS 3.0 RC1 bug fixes

Here are the bugs that have been ‘Fix Released’ in MAAS 3.0 RC1:

Number Description Importance
#1774529 Cannot delete some instances of model ‘Domain’ because they are referenced through a protected foreign key High
#1919001 Unable to network boot VM on IBM Z DPM Partition High
#1925249 MAAS detects 0 cores, RAM available for KVM host, reports negative availability on pod compose High
#1927292 Updating controller has vlan_ids error High
#1927657 Global kernel command line options not passed with tags High
#1928098 If a workload annotation has a key with spaces in it, filtering doesn’t work High
#1926140 maas_url not returned to the UI Medium
#1926171 Failure processing network information when adding a rack Medium
#1927036 Incorrect value “accept_ra” in interface definition Medium
#1927340 Deb to snap migration script should support remote Postgres Medium
#1928104 New workload annotations don’t show up without a reload Medium
#1928115 API still refers to “owner data” rather than “workload annotations” Medium
#1922891 MAAS configures nodes with incorrect DNS server addresses when using multiple IP addresses Undecided
#1923268 grubnet default grub.cfg should try /grub/grub.cfg-${net_default_mac} before /grub/grub.cfg Undecided
#1926164 VLAN page shows odd “Rack controllers” value Undecided
#1926510 dhcp subnet snippets are NOT inside the pool block Undecided
#1927559 Default logical volume size too big in UI Undecided
#1928024 UI states commissioning/testing scripts were never uploaded Undecided
#1928226 Information “not available” indicates that it’'s an error of some sort Undecided
#1928235 notes field won’t update properly: MAAS 3.0 RC]() Undecided
#1928324 updating a machine zone or resource pool doesn’t refresh details Undecided

MAAS 3.0 Beta 5 bug fixes

Here are the bugs that have been Fix Released in MAAS 3.0 Beta 5:

Number Description Importance
#1925784 Processing LXD results failure with loopback Critical
#1923871 LXD vmhost project usage includes usage for other projects High
#1815084 MAAS web ui should perform Save action when Enter/Return is pressed Medium
#1923867 Commissioning fails if NIC gets different PCI address Medium

MAAS 3.0 Beta 4 bug fixes

Here are the bugs that have been Fix Released in MAAS 3.0 Beta 4:

Number Description Importance
#1923246 Unable to compose LXD VM with multiple NICs High
#1918963 Controllers page out of sync with nodes Undecided
#1923685 Unable to deploy LXD VM host on S390X Undecided
#1923687 LXD VM host refresh failure is ignored Undecided
#1774529 Cannot delete some instances of model ‘Domain’ because they are referenced through a protected foreign key High
#1914762 test network configuration broken with openvswitch bridge High
#1919001 Unable to network boot VM on IBM Z DPM Partition High
#1917963 Add chassis lowers the case of added machines Low
#1915087 2.9 UI is broken, seems to loop between user intro and machines pages endlessly High
#1923842 Can’t use action menu on machine details page High
#1917667 Commissioning/testing scripts no longer show ETA or progress Undecided
#1917669 No way to view previous commissioning or testing script results Undecided
#1917670 Storage and interface tests not assoicated with a device Undecided
#1917671 Commissioning/testing scripts not updated after starting commissioning or testing Undecided
#1917794 Unable to view full history of events in UI Undecided
#1918964 UI shows action unavailable after performing action Undecided
#1918966 Tabs aren’t always underscorred Undecided
#1918971 UI does not autofill size on storage tab Undecided
#1923524 Unable to delete LXD composed machine on KVM page Undecided

MAAS 3.0 Beta 3 bug fixes

Here are the bugs that have been Fix Released in MAAS 3.0 Beta 3:

Number Description Importance
#1922569 Create KVM fails in MAAS 3.0 Beta with a project error High
#1923251 Creating an LXD VM host now requires a project name High
#1809939 dhcp snippet create fail when dhcp subnet is relayed Medium
#1913460 Add option to pick whether to keep or decompose machines in a VM host Undecided
#1922787 make “LXD” the default VM host in MAAS UI (rather than virsh) Undecided
#1922876 Deploy KVM hosts with LXD by default Undecided
#1922972 MAAS 3.0 Beta2 UI says “machine cannot be deployed” while successfully deploying machine Undecided
#1923719 MAAS 3.0 : snap refresh maas from 3.0.0~beta2-9826-g.13cc184d5 Undecided

MAAS 3.0 Beta 2 bug fixes

Here are the bugs that have been Fix Released in MAAS 3.0 Beta 2:

Number Description Importance
#1922107 Hugepages/pinning available for virsh and lack validation High
#1922433 Machine resources path set incorrectly in rackd when using snap High

MAAS 3.0 Beta 1 bug fixes

Here are the bugs that have been Fix Released in MAAS 3.0 Beta 1:

Number Description Importance
#1896199 API docs link is not offline Critical
#1904245 MAAS Snap fails to build on PPC64 on Launchpad Critical
#1912727 KVM Page Fails to load with error “An unexpected error has occurred, please try refreshing your browser window.” Critical
#1915869 maas snap cli renders SyntaxWarning in the stderr Critical
#1916093 Unable to add more than 3 Promox VMs Critical
#1883824 Support LXD projects in power control High
#1884276 Terrible user experience adding existing LXD host High
#1902425 Failed to allocate the required AUTO IP addresses after 2 retries High
#1908087 Reverse DNS for non-maas RFC1918 zones fails inside maas High
#1908356 Owner data websocket methods are not working High
#1908434 Can’t delete LXD VM in offline state High
#1913323 /MAAS/docs/ leads to 404 page High
#1914588 Enabling debug from snap traceback High
#1915021 Mapping subnet doesn’t work from the MAAS snap High
#1915022 The MAAS snap doesn’t include nmap High
#1915715 LXD VM additional disks all show 10Gb size High
#1915970 Facebook Wedge BMC detection fails on non-x86 architectures High
#1918997 MAAS does not set snap proxy High
#1919000 Unable to connect MAAS to an LXD VM host High
#1887797 Impossible to delete zombie LXD VM Medium
#1894116 Machines can’t be deployed after deselecting all archs in the “Ubuntu extra architectures” package repo Medium
#1897946 hi1620-based ARM Servers are shown as “Unknown model” Medium
#1906212 timeout in testing scripts ignores the days if set to greater than 24 hours Medium
#1911825 Unable to use FQDN as power_address Medium
#1914165 Proxmox does not allow custom port Medium
#1917652 30-maas-01-bmc-config failing on commissioning Cisco UCSC-C220-M4L Medium
#1335175 maas does not combine kernel_opts when nodes have multiple tags with kernel options Low
#1915359 make sampledata can’t find machine-resources Low
#1916844 Removing a machine that is a vm host tells you to remove the “pod” Low
#1920019 maas_remote_syslog_compress is unnecessarily chatty Low
#1887558 Multipath JBOD storage devices are not shown via /dev/mapper but each path as a single device. Wishlist
#1901944 tags field in machine edit page overtakes other fields Undecided
#1909985 Add commission timestamp to machine websocket api Undecided
#1913464 Drop RSD pods UI Undecided
#1914590 Support composing LXD VMs with multiple disks in the UI Undecided
#1915970 Facebook Wedge BMC detection fails on non-x86 architectures Undecided
#1916073 MAAS should install qemu-efi-aarch64 on arm64 KVM pods Undecided
#1916317 UI is using API to request scripts with full content Undecided
#1919381 typo “veryiying” in info message in smartctl-validate Undecided

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.