What happens to Wake-on-LAN on machine-shutdown during "Enlistment" and "Commissioning"?

Users
1

Setting aside any topics such this and this, I’m instead curious about something very specific as it relates to MAAS:

When we manually construct a machine/node (i.e. put together the motherboard, CPU/RAM/storage, etc.), we also provision a UEFI-settings snapshot to all nodes. This also ensures that “Wake-on-LAN” (WoL) is enabled, uniformly, from the beginning.

Observations

  • Before MAAS even knows of a node, we power it on manually and run a simple manufacturer’s diagnostic
  • We then emit a WoL “magic packet” from the console of the MAAS-hosting VM - and it works
  • After MAAS sees the node for the first time - Enlistment - it finishes that first touch by shutting down the node
  • We then emit a WoL “magic packet” from the console of the MAAS-hosting VM - and it doesn’t work

In this outcome-condition, we have to then manually power on the node. When “Commissioning” a node, the same occurs: WoL is active/working (by virtue of having turned on the node by hand), MAAS begins “Commissioning” phase, MAAS finishes by shutting down the node - and WoL no longer works.

It’s understood that despite the UEFI/BIOS setting of having WoL enabled, Ubuntu itself must also be configured/enabled (ethtool, etc.) to make WoL persist between reboots/shutdowns.

Questions

With the above in mind:

  • What is MAAS doing as part of Enlistment/Commissioning such that WoL “doesn’t work” after the requisite “shutdown” when each phase completes?
  • Akin to “need to configure Ubuntu to make WoL persist after reboot/shutdown” - does that opportunity already exist when MAAS is doing Enlistment/Commissioning? (read: “are we just missing something here on our end?”)
  • Adding to the above: is Ubuntu “already on the node” at the point of Enlistment/Commissioning? We didn’t think so, but… maybe? (was under the impression that “making WoL persist, as a setting” required the OS - Ubuntu - to exist. Maybe not? Like, how else would WoL not persist only after MAAS has had its first touch on a node? mea culpa)

Essentially we’re just wondering if we’re oblivious to some obvious config/setup/setting we could be employing in MAAS (some… Enlistment/Commissioning-phase script?) that would allow pre-Deployment-OS-presence to allow WoL to… persist/stick/be “on”/not cease to be/not pine for the fjords.

Any help would be greatly appreciated. Figuring this out would allow MAAS to be, for us, finally “from-across the room”-able.

2

Enlistment works by having a data center tech manually power on the system after it has been configured to netboot. MAAS collects some basic information and then powers it off. This should work with WoL.

When a machine is commissioned MAAS will reboot it if its already turned on. This isn’t possible with WoL. If the machine is off WoL could start the machine and allow commissioning to run. Both commissioning and enlisting boot Ubuntu from over the network. Neither enlisting nor commissioning modify the system in anyway. You can add custom commissioning scripts which allow you to do anything you want. For example you could add a commissioning script which enables WoL.

Deploying an operating system does change the boot order. On UEFI systems the installed OS is configured to boot after the network card. I’m not sure why WoL would stop working but when I’ve used it in the past its very finicky.

The problem with WoL is that it only allows you to power on a machine. MAAS needs to be able to check the power status, turn the machine off, and reboot it if necessary.

1 Like
3

@ltrager - Thanks for taking the time to respond! We’re certainly hopeful that a simple “WoL” option returns at some point in the future, even fully acknowledging it’s an “on-only” option. Still, understood on the position at this time :slight_smile:

You can add custom commissioning scripts which allow you to do anything you want. For example you could add a commissioning script which enables WoL.

I have a simple service file created. Is this what we would use? The documentation indicates this is possible, but it’s a bit unclear as to how we’d go about accomplishing this. If the OS is not yet on the host at the time of Enlistment, after Enlistment, at the time of Commissioning, and after Commissioning, how would WoL be enabled at the software level, equivalent to the service file we’ve written for the OS?

Pardon any potential ignorance here. I’m learning :slight_smile:

4

WoL is actually a setting in firmware so the OS doesn’t matter. If I had to guess the service file you have checks that the WoL setting is still on.

Currently MAAS doesn’t support custom scripts during enlistment but that is being added to MAAS 2.9.

5

Thanks for the follow-up @ltrager!

To clarify: we’re on the same page regarding the need for our hardware to support WoL, and that the UEFI/BIOS tier of things must also then be configured such that WoL (for that interface) is enabled. We agree! :slight_smile:

We believe there still may be more to it than this, when it comes to MAAS/perhaps Ubuntu in general. We found this support article from Ubuntu’s community wiki, and we agree with at least one of the sentiments expressed there (strongly attempting to set aside any confirmation-bias, of course):

sudo ethtool -s <NIC> wol g
On most systems, issuing this command is required after each boot

Most notably, the seemingly-still-active/open bug in Launchpad seems to orbit what we’re experiencing: Launchpad - Bug 981461

Why we find all of this interesting: when a node is brought online - assembled physically, without any prior OS installation, and without any interaction with MAAS - we enable WoL in UEFI BIOS, physically shut-down/power-up the node (which engages WoL), and then physically shut down the node. We then confirm WoL is working by issuing a “magic packet” (wakeonlan) to that node. It works! Nice :slight_smile:

We then enable PXE boot in boot order - confirming WoL is still “active” - and then let MAAS take the wheel.

It is that that point that once MAAS finishes its initial “Enlisting” phase, such that the node shuts down (power-off event), that WoL stops functioning.

We then physically power on the node, intercept the prompt for UEFI/BIOS Settings (F2), and then confirm that PXE (IPv4) is still enabled - and it is. Again, nice :slight_smile:

However, as mentioned above, the node won’t wake up via WoL, following the exact same process before (emitting magic packet to MAC address, etc. etc.). It’s not until we get through Commissioning and then Deployment that we can then enable it on the OS-tier so (see the above article I linked).

Hopefully this helps apply some clarity to what we’re encountering/curious about. Thanks again for your time!

6

Use Ubuntu 14.04 as default OS for Enlisting and the WoL feature will be present on the node. (You can then power up the node using a magic packet)

7

@rochajoel - Is there any way to do this with 20.04?

It seems you’ve touched on what I suspected - that somehow, Enlistment/Commissioning modifies whatever right be akin to Ubuntu’s wol g args for an interface.

8

Hi, facing the same issue… but dug a little deeper.

It is the Linux kernel that does this. There is a mailthread about this, the main issue is that the kernel doesn’t know if its on or off, so since the latest kernels its by default OFF. When you enable WOL and shutdown you can send a magic packet and it will turn on, however by booting the kernel it will turn off again.

Honestly it feels a little stupid, but on the other hand i can understand it as it would be enabled by default the other way which most people don’t want with their PC. But after enlisting/commisioning you will have the same problem unless you use systemd or something to set it to enabled once the system is booted up.

I would love for the dev’s of MAAS to add the enablement of WOL by default in the newer images, looking at the upcoming 2.9 version where you can customize enlist and commision scripts you can do it like that too with newer OS like 20.04 … but i dont know the release date … or found a working hack to modify the enlist image manually … for the commision part adding a small script enabling WOL works.

Using the 16.04 version is not a solid solution, as it might not turn off WOL, but it does not enable it either … as such you are still stuck getting WOL enabled automatically in the first place.

9

With Ubuntu guests at least 18.04 and above I think, the solution seems to be having the netplan.yaml file set with MAC addresses of each interface, and then the wakeonlan parameter set to true. Forgive me if I’ve missed anything else needed for WoL in the YAML, but if I did, just add those in too. Nonetheless, if you boot a machine with the appropriate netplan YAML file in place, and shut it down / reboot it correctly, it should maintain the WoL across power cycles. Example seen here of what’s needed for WoL in the YAML file. https://ubuntuforums.org/showthread.php?t=2394211 I’m sure this whole process for generating or having those YAML files in place can be templated or dynamically generated, e.g. MAC addresses.

10

AFAIK, we don’t support WOL for MAAS.

11