(Maas 2.6.2) Maas always seems to do a “apt-get dist-upgrade” as part of deploying a machine, which is a major problem for us as we use an internal aptly snapshot of upstream that “lags” behind upstream by a certain amount of time (so that all machines can be patched to a CONSISTENT source). The problem is that maas won’t allow me to direct it at our static snapshot as its “ubuntu mirror”, because the snapshotting/aptly process ends up changing some fields in the InRelease/Release files which causes cloud-init in the ephemeral image to choke on and immediately abort, so I’m forced to leave maas configured to point at upstream, the problem being when maas deploys a machine, some hidden compiled in step within curtin/maas does an “apt-get upgrade” to the box while pointing to the upstream mirrors which results in pulling down packages NEWER than our delayed snapshot. This results in sporadic APT failures AFTER deployment when the machine is pointed back at our static snapshots that are now “behind” (by design). i.e. apt-get upgrade during deploy updates libc, and a process on some VM’s after deployment needs to install libc6-dev which now pointing at our static snapshot is behind the libc that got updated (unwantedly by maas) which results in an apt failure and a manual forced-downgrade of libc is necessary to resolve the situation.
We build our images to be CONSISTENT and mass’s blind “apt-get upgrade” during deployment completely breaks that.
Looking for one of the following:
A way to turn off apt-get upgrade by maas during the deployment process (via api call or gui)
Or some magic flags to cloud-init that I can feed via curtin to make it ignore the fields in InRelease/Release that cause it to complain when in the ephemeral (We have no issues with all of our machines that point to it today, its somehting specifically in cloud-init’s apt handler than chokes on it)
1 Like
If there isn’t already a way to do this, I would definitely support it. Essentially, curtin needs to be told not to do the update/upgrade, and there needs to be a flag in the API and possibly the deployment GUI to tell MAAS to use that curtin capability.
1 Like
MAAS/Curtin don’t update deployments by default. You can manually configure a deployment to do this using a Curtin preseed or cloud-init user_data.
apt has to run during an Ubuntu deployment to install the kernel and bootloader. The Ubuntu image MAAS uses is the base SquashFS image Canonical provides for all clouds. This image does not contain a kernel or bootloader as not all clouds require them(e.g LXD) and to allow selecting a specific kernel during deployment time. Curtin runs apt update to get the repository metadata cache. It then installs the user selected kernel and any bootloaders the architecture requires. If the kernel or bootloaders require newer packages they are automatically updated, other packages are left unchanged.
We build our images WITH A SPECIFIC KERNEL and configuration, we do not want maas to override that. I’ve already had to put in place logic to undo what maas is doing with regards to the kernel. The whole point of an image based install is to have a LOCKED configuration blown onto machines so they are all 100% consistent. With the current maas behavior, over a period of a week or a few days or a month, each machine will end up with a different kernel and some newer packages than our static repo snapshot which defeats the purpose of a “consistent image”.
MAAS does not currently support custom Ubuntu images. See Creating a custom Ubuntu image for the reasons why as well as a suggested method to create a custom image. Unfortunately the kernel is part of the reason we don’t support custom Ubuntu images.
The supported method to perform Ubuntu deployments with a static set of packages is to mirror official images, Ubuntu archives, and configure MAAS to use your mirror. Once configured MAAS will reconfigure the image to use your mirror in the ephemeral environment and during deployment. You can then control when the mirror is updated to ensure the package set is static.
From the detailed logs of a machine during deployment:
Node installation - ‘curtin’ updating packages on target system
It IS updating things and has lead to APT issues after deployment due to it updating during the installation process.
A custom ubuntu image is needed and built in-house because it has our customizations for single sign on, basic puppetization, security suite installed, etc.
We tried a custom mirror, and cloud-init in the ephemeral image chokes on it and powers down the host because aptly mangles some of the fields in the InRelease/Release files. We’re working on trying to resolve that, though aptly isn’t very flexible in that regard.
