MAAS 2.6.0 alpha 1 released!

andreserl · 13 March 2019 15:30

Hello MAASters!

We are happy to announce MAAS 2.6.0 alpha 1. This release introduces new and exciting features and UI improvements.

Important announcements

Deploying machines with multiple “default” gateways

Deploying machines in complex network scenarios, where you want traffic to go out through a different gateway other than the default gateway, requires machines to be configured with source routing. This ensures that any outgoing traffic for which there are no defined routes goes out through the same interface as the incoming traffic.

MAAS 2.6 now changes the default machine network configuration to take advantage of source routing.

New features & improvements

Multiple default gateways

Previous MAAS versions deployed machines with a single default gateway. This meant that if a machine was configured in multiple subnets with gateways (with or without any routes), all incoming traffic on a given gateway would go out through the default gateway instead of the intended gateway.

From MAAS 2.6, this behavior now changes as MAAS now leverages the use of netplan and source routing to configure multiple “default” gateways, ensuring that all outgoing traffic goes out through the correct gateway rather than always going out through the default gateway.

For example, in previous versions of MAAS if a machine was with two subnets (subnet1, subnet2), and using subnet1’s gateway as the default gateway, all traffic in subnet two to the outside world would go out through the default gateway. Instead, with the newer changes, all traffic in subnet2 will go out through subnet2’s gateway in case it needs to reach the outside world.

Ephemeral deployment for disk-less machines

In older versions, MAAS prevented machines without any attached disks to be deployed. As of 2.6, MAAS now allows deployment all diskless machines to be ephemerally deployed, by default, which simply runs the operating system in memory. This feature provides the ability to:

Network interface configuration - MAAS will apply the network interface configuration on the ephemeral environment.
Deployment customization via cloud-init (user-data) - MAAS will continue to allow users to provide user-data to customize their deployments.
Reconfiguration on reboot - On reboot, an ephemerally deployed machined will re-run all configuration that was provided to cloud-init, including that provided by the user (user-data).

Please note that ephemeral deployments are limited in that:

Ubuntu as the only ephemeral OS.
Curtin preseeds are no longer applicable for deployment configuration.
Storage is no longer configured.

Redfish power management

MAAS now adds the ability to power manage machines via Redfish, and has been tested against HP and Dell servers. Please note that most BMC’s continue to support IPMI alongside Redfish REST APIs for remote management of the BMC, as such a few considerations:

If the machine is automatically enlisted, MAAS will default to discover/configure IPMI.
If a machine has been switched to use Redfish, or created with the Redfish power driver, MAAS will always use this and won’t automatically switch back to IPMI.
It allows defining a node ID in case the machine is inside a chassis management interface with multiple machines.

PXE over UUID

MAAS now supports legacy PXE booting over UUID. The MAAS commissioning process will now gather the machine’s UUID, which is also used for the PXE process over pxelinux. MAAS, by default, will now prefer to respond to PXE requests based on the UUID rather than the MAC address. This was originally added to start supporting s390x deployment.

Minor Performance improvements

As a result of a bugfix where MAAS was creating database records for interface links against null IP address, the UI performance has been increased. This underlying issue was causing load times of network elements in the UI. The fix has reduced the queries from 30s to 1s. This improves the perceived performance in large environments where there’s significant DHCP traffic.

Work in progress

Apart from the given features, MAAS has other various features that are being develop concurrently. These include:

Stats and metrics gathering

MAAS is adding support to start collecting metrics and stats about the environment and performance. This will all users to have more insight about usage and performance, as well as help developers use this data to futher improve performance.

ESXi - Storage support creating data stores

The current ESXi support only allows for configuring the installation of the ESXi image on a single disk. In 2.6, MAAS will support the ability to further configure storage by providing the ability to configure extra data stores for the VMs.

Web UI

Bulk tagging

MAAS 2.6 introduces the ability to tag machines in bulk over the Web UI.

Clear discovered devices

The device discover page now has the ability to clear all discovered records, or clear individual ones.

Power actions in table (machine listing)

The machine listing page now includes an in-table dropdown action to execute power actions on the machine.

Bug fixes

For other issues or bugs fixed in this release, please refer to:

https://launchpad.net/maas/+milestone/2.6.0alpha1

Availability

sudo apt-add-repository ppa:maas/next
sudo apt install maas

tmartins · 14 March 2019 00:20

I can’t wait to give it a try!

Today, none of my MaaS’ networks have gateway configured and I have to manually change Netplan after the deployment, to enable the gateway that I want.

Also, the MaaS “Netplan Generator” is unreliable. Depending on how I configure the interfaces, the deployed Machine have no Netplan file, I can see an “netplan generator ERROR” (or something like that) and the deployment bootstrap fails in the end.

Then, I boot into rescue mode and the /etc/netplan subdir is empty.

Couple months ago I found a bug on it as well, while trying to deploy a Machine as KVM host. AutoAssign was failing, only StaticIP worked.

So, i hope that MaaS 2.6 will bring a much more professional and stable Netplan generator!

I would love to fully trust on MaaS but, these days, it’s impossible to trust it. I mean, I use bare minimum functions of it, for example:

I would like to:

Use MaaS DNS for my entire infra but, I can’t, if you overlap a subnet, MaaS will break Bind9 configs and the process will not start anymore. So, I can not trust maas-dns. I have to keep 2 extra DNS Servers elsewhere, especially because I’m about to activate OpenStack Designate.
Use MaaS DHCP for non-PXE boot networks as well, like as a DHCP Relay Server (is it possible?), but, again, I don’t trust it that much. So, again, I have to keep a separated isc-dhcp-server in a VM and it’s a SPOF right now (working to have 2 DHCP Relay Servers and keepalived/VIP).
It isn’t possible to manually update the Machine’s Netplan and tell MaaS about the changes… This is especially bad.
MaaS Pod Compose function is super limited. I can’t choose to compose a Machine using UEFI, needed for the new “zfsroot” option. It also hides many options that are easily available on Virt-Manager, like copy CPU host features, virtio-vga, virtio-scsi and etc. So, today, I don’t use MaaS Compose, but instead, Virt-Manager and then, I tell MaaS to “Refresh the Pod”, so I can deploy Ubuntu easily…
No support for Debian, FreeBSD or PFSense. Also no support to add those extra O.S. Images… Which sucks. Today, if I need a PFSense VM, I have to deploy Ubuntu with all the required network interfaces and then, via Virt-Manager, I manually reinstall the VM with PFSense ISO. At least, MaaS knows about the allocated IPs…
No support for deploying Machines in RAID1 where the two devices are bootable. This is something that MUST be done ASAP! Especially for EFI/GPT Machines. But MaaS doesn’t do that even for BIOS/MBR…

I hope that Canonical will invest 10 times more money into MaaS, because it’s ideas are awesome! But the implementation is buggy, unstable and unreliable.

MaaS FTW!

Cheers!

tmartins · 14 March 2019 14:58

I also think that it would be awesome to have support for:

MACVLAN;
OpenvSwitch (both Kernel and DPDK).

tmartins · 15 March 2019 00:43

I would be nice to bring QCoW2 support for Composed KVM Guests! Because with RAW Images, there is no way to take snapshots of the VMs and this is a must to have!

Another reason why I don’t use MaaS’ Compose function.

seffyroff · 15 March 2019 08:02

Did a DB column get renamed? Upgraded my dev 2.5.x region server to 2.6a1. Rack came back up fine, region seems unhappy:

2019-03-15 08:00:14 maasserver.start_up: [error] Database error during start-up
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: column maasserver_node.hardware_uuid does not exist
LINE 1: ...r_node"."updated", "maasserver_node"."system_id", "maasserve...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/maasserver/start_up.py", line 74, in start_up
    yield deferToDatabase(inner_start_up, master=master)
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 894, in callInContext
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 235, in wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 712, in call_with_connection
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/__init__.py", line 192, in call_with_lock
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 756, in call_within_transaction
    return func_outside_txn(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 563, in retrier
    return func(*args, **kwargs)
  File "/usr/lib/python3.6/contextlib.py", line 52, in inner
    return func(*args, **kwds)
  File "/usr/lib/python3/dist-packages/maasserver/start_up.py", line 116, in inner_start_up
    region = RegionController.objects.get_or_create_running_controller()
  File "/usr/lib/python3/dist-packages/maasserver/models/node.py", line 745, in get_or_create_running_controller
    node = Node.objects.get(system_id=maas_id)
  File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 374, in get
    num = len(clone)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 232, in __len__
    self._fetch_all()
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1118, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 899, in execute_sql
    raise original_exception
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 889, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/lib/python3/dist-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column maasserver_node.hardware_uuid does not exist
LINE 1: ...r_node"."updated", "maasserver_node"."system_id", "maasserve...
                                                             ^

ubuntu@juju-3d0654-1:~$

andreserl · 15 March 2019 09:10

Hey there,

Thank you for raising the issue, but please file a bug report (https://bugs.launchpad.net/maas/+filebug) as that’s really the most appropriate place to discuss potential issues. Please also provide relevant logs:

dpkg -l | grep maas
/var/log/maas/term.log
/var/log/maas/*.log
ls -l /usr/lib/python3/dist-packages/maasserver/migrations/builtin/maasserver/

seffyroff · 15 March 2019 15:49

Yep will do, I posted here initially wondering if it was something straightforward like a dbupdate script I needed to run…

djcutch · 18 March 2019 18:37

Hello,

Did your team finish the tool for esxi deployment to start testing all these new features.

as i am trying to import esxi 6.7 image with the following command

maas admin boot-resources create -d name=esxi architecture=amd64/generic content@=$HOME/ESXi.tgz
400 Bad Request

Content-Length: 129
Content-Type: application/json; charset=utf-8
Date: Wed, 20 Mar 2019 03:49:40 GMT
Server: TwistedWeb/17.9.0
Status: 400
Vary: Cookie
X-Frame-Options: SAMEORIGIN

{“name”: [“esxi is a reserved name”], “architecture”: ["‘amd64/generic’ is not a valid architecture. It should be one of: ‘’."]}

tmartins · 27 March 2019 03:54

I’m wondering here… Will this new Diskless mode be similar to the today’s “Rescue Mode”?