HTTP boot in MAAS

Users
1

MAAS supports booting over TFTP or HTTP – whatever your firmware supports. On every platform except S390X, once the boot loader has transferred, the kernel and initrd are always transferred over HTTP, regardless of boot method.

MAAS attempts to use HTTP during the boot process as much as possible. In the case of legacy PXE booting, the boot loader is transferred over TFTP, while all remaining resources are transferred over HTTP. The UEFI specification allows for booting to happen over TFTP and, in later editions, HTTP. The bootloader (grub) and boot loader configuration will be transferred using whatever protocol your firmware supports. All remaining files will be transferred over HTTP. MAAS currently doesn’t support HTTPS.

Here’s a detailed rundown of different combinations:

  • PXE Booting(BIOS) - Legacy x86_64/i386 systems only support booting over TFTP. Newer versions of pxelinux ship with lpxelinux.0 which enable HTTP support in pxelinux. During PXE booting the machine gets lpxelinux.0 over TFTP; pxelinux.cfg, the kernel, initrd, and SquashFS are all transferred over HTTP.

  • PXE Boot VM(BIOS)/iPXE - kvm implemented legacy PXE booting by using iPXE. No bootloader needs to be sent if the system is using iPXE. Instead, DHCP tells iPXE where to get its config file. MAAS does not provide the iPXE bootloader, but it will generate iPXE config if requested. Note that ipxe.cfg is transferred over HTTP; TFTP isn’t used at all with iPXE.

  • UEFI Netboot TFTP - Many older UEFI systems only support TFTP booting. Others, like OVMF (firmware kvm uses for UEFI), it tries TFTP first, then HTTP. MAAS sends grub over TFTP, which requests its config over TFTP.

  • UEFI Netboot HTTP - MAAS supports full HTTP boot via UEFI (if the firmware supports it).

  • UEFI Netboot HTTPS - MAAS does not currently support HTTPS booting.

2 Likes
2

With respect to EFI, the process more involved: [current thinking is that] Everything
is transferred over TFTP. It’s possible that a grub update is necessary to enable transfer of kernel, parameters, and initrd over HTTP.

When UEFI boots over TFTP only GRUB and grub.cfg are transferred over TFTP. The kernel and initrd are transferred over HTTP. The grub bug that was mentioned was fixed years ago. You can confirm this by looking at the current grub.cfg being handed out

$ curl tftp://10.0.0.2/grub/grub.cfg-default-amd64
set default="0"
set timeout=0
menuentry 'Commission' {
    echo      'Booting under MAAS direction...'
    linuxefi  (http,10.0.0.2:5248)/images/ubuntu/amd64/ga-18.04/bionic/daily/boot-kernell nomodeset ro root=squash:http://10.0.0.2:5248/images/ubuntu/amd64/ga-18.04/bionic/daily/squashfs ip=::::maas-enlist:BOOTIF ip6=off overlayroot=tmpfs overlayroot_cfgdisk=disabled cc:\{'datasource_list': ['MAAS']\}end_cc cloud-config-url=http://10-0-0-0--24.maas-internal:5248/MAAS/metadata/latest/enlist-preseed/?op=get_enlist_preseed apparmor=0 log_host=10.0.0.2 log_port=5247 BOOTIF=01-${net_default_mac}
    initrdefi (http,10.0.0.2:5248)/images/ubuntu/amd64/ga-18.04/bionic/daily/boot-initrd
}

There is also the UEFI HTTP boot process (not yet widely supported), in which everything is transferred over HTTP.

I disagree with this. Its been part of the UEFI spec since 2015. New hardware should support this. Most, if not all of the MAAS CI hardware is from before 2015 which is why we don’t have access to test it.

MAAS contains untested code which supports UEFI over HTTP (no test hardware),

Blake tested with on hardware(I think the EA hardware but I don’t remember). I tested this with kvm but it has been awhile.

but most venders require HTTPS

I don’t think this is true. Infact if you read the UEFI spec it states that if the firmware support HTTPS it must also support HTTP.

If a platform includes the ability to perform a HTTP-based boot from a network device, then the EFI_HTTP_SERVICE_BINDING_PROTOCOL,EFI_HTTP_PROTOCOL and EFI_HTTP_UTILITIES_PROTOCOLare required. If it includes the ability to perform a HTTPS-based boot from network device, besides above protocols, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL are also required. If it includes the ability to perform a HTTP(S)-based boot with DNS feature, then EFI_DNS4_SERVICE_BINDING_PROTOCOL, EFI_DNS4_PROTOCOLare required for the IPv4 stack; EFI_DNS6_SERVICE_BINDING_PROTOCOL and EFI_DNS6_PROTOCOL are required for the IPv6 stack.

3

When UEFI boots over TFTP only GRUB and grub.cfg are transferred over TFTP. The kernel and initrd are transferred over HTTP. The grub bug that was mentioned was fixed years ago. You can confirm this by looking at the current grub.cfg being handed out

Oh, that’s news to me, and I’m glad to hear it At least when I did the performance testing for 2.6.0, I seem to remember that EFI booting caused initrd and kernel to be transferred over HTTP. But maybe that was what I was told, I can’t remember if I actually tested it.

In fact, that was the reason why we wanted to replace the TFTP server. All the other files are so small, so transferring them over TFTP is not a problem

I disagree with this. Its been part of the UEFI spec since 2015. New hardware should support this. Most, if not all of the MAAS CI hardware is from before 2015 which is why we don’t have access to test it.

Sure, it’s in the spec, but it’s not very clear. Everyone seems to have their own interpretation of it.

Also, just because it’s in the spec, vendors don’t need to support it. For example, my laptop is fairly new, supports UEFI, but it doesn’t support HTTP boot. Not sure what the state is for server vendors, but I wouldn’t assume that most support it, unless we see some numbers. And sure, I’m also not assuming that most don’t support it. What I do know is that we (the MAAS team) haven’t tested it much, and we can’t say whether it works or not. If you successfully tested it on KVM, it means that so far we’ve had a 50% success rate over 4 different systems.

Blake tested with on hardware(I think the EA hardware but I don’t remember). I tested this with kvm but it has been awhile.

When I tried to debug an EFI HTTP boot issue, Andres and Blake clarified that they had tested it on two systems. One stopped working after a firmware update. They also couldn’t give me access to another system that supporeted EFI HTTP boot. So with the system I had, we had tested it on a total of 3 system, and only 1 of them worked.

The one I tested required HTTPS. They didn’t bother debugging why the firmware update broke things for the other system, but I wouldn’t be surprised if it was a similar reason.

I don’t think this is true. Infact if you read the UEFI spec it states that if the firmware support HTTPS it must also support HTTP

The section you quoted seems to be more of an internal API. For HTTP booting the firmware need modules X, and for HTTPS you need modules X and Y, where Y is only dealing with TLS and then use the X modules for the HTTP part.

There’s nothing in the spec that says that a firmware can’t have a policy of HTTPS only. You might understand the spec in that way. That brings me back to the point that the spec is unclear, and everyone understands it differently.

4

Couple of notes on the update by Bill

  • I would simply state that MAAS supports booting over TFTP or HTTP, whatever your firmware supports. Users don’t really care what the state of different vendors, they just want their hardware to work.
  • I’m not sure how much we want to reveal WRT to what hardware we’re testing with.
  • I don’t think showing the grub output really adds anything. I wrote a discourse post before you joined about it - MAAS Internal URLs
  • On every platform except S390X once the boot loader loads the kernel and initrd are transferred over HTTP.
5

I agree with Lee’s comment.

I also think that we should make this document about “Network booting”, and not HTTP specific. Having a diagram would be useful to show that different steps of the process.

6

@bjornt, i can do diagrams! :slight_smile:

7

the kernel and initrd (the Ubuntu SquashFS image) is always transferred over HTTP

The initrd and SquashFS are two separate things. The SquashFS is downloaded by the initrd which is why it is always transferred using HTTP.

HTTP is supported well for legacy PXE booting. In that case, the PXE binaries are transferred and configured over TFTP. With respect to EFI, the process slightly more involved: GRUB and grub.cfg, which are very small files, are transferred over TFTP. In the UEFI boot model, everything is transferred over HTTP. MAAS supports UEFI over HTTP.

I feel like this is mostly repeated below so you could remove it or replace it with this

MAAS attempts to use HTTP during the boot process as much as possible. In the case of legacy PXE booting the boot loader is transferred over TFTP while all remaining resources are transferred over HTTP. The UEFI specification allows for booting to happen over TFTP and in later editions HTTP. The bootloader(grub) and boot loader configuration will be transferred using whatever protocol your firmware supports. All remaining files will be transfered over HTTP.

We currently cannot give any concrete answer on whether a machine supports HTTP or HTTPS, as firmware may or may not have an HTTPS-only policy.

MAAS doesn’t currently support HTTPS at all. I’m not sure I would mention anything more then that.

Also I believe in Bjorn’s case the machines that had an HTTPS only policy still supported TFTP which MAAS does support.

During PXE booting the machine gets lpxelinux.0 over TFTP.

During PXE booting the machine gets lpxelinux.0 over TFTP. pxelinux.cfg, the kernel, initrd, and SquashFS are all transfered over HTTP.

PXE Boot VM(BIOS)/iPXE - kvm implemented legacy PXE booting by using iPXE. No bootloader needs to be sent if the system is using iPXE. Instead, DHCP tells iPXE where to get its config file. MAAS does not provide the iPXE bootloader, but it will generate iPXE config if requested.

ipxe.cfg is transfered over HTTP, TFTP isn’t used at all with iPXE.

MAAS sends grub over TFTP, which requests its config over TFTP.

all remaining files are sent over HTTP.

UEFI Netboot HTTP - All modern UEFI systems do support HTTP boot. On machines that require HTTPS, it may be possible to disable secure boot. KVM UEFI supports HTTP boot, but you have to reorder your boot configuration with efibootmgr, since KVM UEFI attempts TFTP before HTTP. MAAS supports full UEFI HTTP boot.

I feel like this may be getting to much into the details. I would say something like

UEFI Netboot HTTP - MAAS support full HTTP booting using UEFI if the firmware supports it.

** UEFI Netboot HTTPS** - MAAS does not currently support HTTPS booting.

8

Comments incorporated into latest post, @ltrager, @bjornt.