CentOS 9 Stream build for Maas

Users
,
1

Anyone here has been able to successfully build CentOS 9 Stream for Maas using packer? Below is the error that I am getting. I am unsure if this is completely unsupported at the moment or am I doing something incorrect with the build.

2023/05/08 21:00:37 [INFO] (telemetry) Starting post-processor shell-local
==> qemu.centos9-stream: Running post-processor: (type shell-local)
2023/05/08 21:00:37 packer-post-processor-shell-local plugin: [INFO] (shell-local): Prepending inline script with #!/bin/bash -e
==> qemu.centos9-stream (shell-local): Running local shell script: /tmp/packer-shell726199497
2023/05/08 21:00:37 packer-post-processor-shell-local plugin: [INFO] (shell-local): starting local command: /bin/sh -c PACKER_BUILDER_TYPE=‘qemu’ PACKER_BUILD_NAME=‘centos9-stream’ PACKER_HTTP_ADDR=‘10.0.2.2:8814’ PACKER_HTTP_IP=‘10.0.2.2’ PACKER_HTTP_PORT=‘8814’ /tmp/packer-shell726199497
2023/05/08 21:00:37 packer-post-processor-shell-local plugin: [INFO] (shell-local communicator): Executing local shell command [/bin/sh -c PACKER_BUILDER_TYPE=‘qemu’ PACKER_BUILD_NAME=‘centos9-stream’ PACKER_HTTP_ADDR=‘10.0.2.2:8814’ PACKER_HTTP_IP=‘10.0.2.2’ PACKER_HTTP_PORT=‘8814’ /tmp/packer-shell726199497]
qemu.centos9-stream (shell-local): Loading nbd

qemu.centos9-stream (shell-local): Using /dev/nbd13
qemu.centos9-stream (shell-local): Binding image to /dev/nbd13

qemu.centos9-stream (shell-local): /dev/nbd13 disconnected
qemu.centos9-stream (shell-local): Waiting for partitions to be created

qemu.centos9-stream (shell-local): Disk /dev/nbd13: 4 GiB, 4294967296 bytes, 8388608 sectors
qemu.centos9-stream (shell-local): Units: sectors of 1 * 512 = 512 bytes
qemu.centos9-stream (shell-local): Sector size (logical/physical): 512 bytes / 512 bytes
qemu.centos9-stream (shell-local): I/O size (minimum/optimal): 512 bytes / 512 bytes
qemu.centos9-stream (shell-local): Mounting root partition

==> qemu.centos9-stream (shell-local): mount: /tmp/packer-maas-7UEm: special device /dev/nbd13p1 does not exist.
==> qemu.centos9-stream (shell-local): umount: /tmp/packer-maas-7UEm: not mounted.
2023/05/08 21:01:37 packer-post-processor-shell-local plugin: error: Script exited with non-zero exit status: 32. Allowed exit codes are: [0]
2023/05/08 21:01:37 [INFO] (telemetry) ending shell-local
2023/05/08 21:01:37 Deleting original artifact for build ‘qemu.centos9-stream’

  • Post-processor failed: Script exited with non-zero exit status: 32. Allowed exit codes are: [0]
    Build ‘qemu.centos9-stream’ errored after 6 minutes 3 seconds: 1 error(s) occurred:

==> Wait completed after 6 minutes 3 seconds
2023/05/08 21:01:37 machine readable: error-count []string{“1”}
==> Some builds didn’t complete successfully and had errors:
2023/05/08 21:01:37 machine readable: qemu.centos9-stream,error []string{“1 error(s) occurred:\n\n* Post-processor failed: Script exited with non-zero exit status: 32. Allowed exit codes are: [0]”}

  • Post-processor failed: Script exited with non-zero exit status: 32. Allowed exit codes are: [0]

==> Wait completed after 6 minutes 3 seconds

  • Post-processor failed: Script exited with non-zero exit status: 32. Allowed exit codes are: [0]
    ==> Builds finished but no artifacts were created.
    2023/05/08 21:01:37 [INFO] (telemetry) Finalizing.

==> Some builds didn’t complete successfully and had errors:
–> qemu.centos9-stream: 1 error(s) occurred:

  • Post-processor failed: Script exited with non-zero exit status: 32. Allowed exit codes are: [0]

==> Builds finished but no artifacts were created.
2023/05/08 21:01:37 waiting for all plugin processes to complete

2023/05/08 21:01:37 /root/.config/packer/plugins/github.com/hashicorp/qemu/packer-plugin-qemu_v1.0.9_x5.0_linux_amd64: plugin process exited
2023/05/08 21:01:37 /usr/bin/packer: plugin process exited
make: *** [Makefile:8: centos9-stream.tar.gz] Error 1

2

Hey @johnliaotvc, I’ve managed to create a Centos9 image. I’ll share what I’ve done during my evening (in some hours)

3

Hi @johnliaotvc , could you take a look at https://github.com/canonical/packer-maas/pull/111 and double check if that works for you?

1 Like
4

Thanks for the quick reply. I was able to build the image successfully. However on boot, it is reporting the error below. My custom image on the maas server is located in this directory.

http://10.16.102.47:5248/images/centos/amd64/generic/9-custom/uploaded/root-tgz

According to the error message, it seems to be trying to boot of from another directory. Is this control by cloud-config somewhere?

Thanks

Booting from Hard drive C:

Booting from IBA 40G Slot 8100 v1130

IntelÂź Boot Agent 40G v1.1.30
Copyright © 1997-2020, Intel Corporation

PXELINUX 6.04 lwIP 20191223 Copyright © 1994-2015 H. Peter Anvin et al
Booting under MAAS direction

nomodeset ro root=squash:http://10.16.102.47:5248/images/centos/amd64/ga-20.04/9
-custom/no-such-image/squashfs ip=::::srv-01-11-406:BOOTIF ip6=off overlayroot=t
mpfs overlayroot_cfgdisk=disabled cc:{‘datasource_list’: [‘MAAS’]}end_cc cloud-c
onfig-url=http://10-16-96-0–20.maas-internal:5248/MAAS/metadata/latest/by-id/ec
7xaa/?op=get_preseed apparmor=0 log_host=10.16.102.47 log_port=5247 —
Loading http://10.16.102.47:5248/images/centos/amd64/ga-20.04/9-custom/no-such-i
mage/boot-kernel
 failed: No such file or directory
boot:

5

http://10.16.102.47:5248/images/centos/amd64/ga-20.04/9-custom/no-such-image/squashfs looks a bit scary :sweat_smile: I’ll try to take a look tomorrow

1 Like
6

what version of MAAS are you using? I’m not able to reproduce your issue on our main branch

7

I am on the latest version 3.3.2.

maas admin version read

Success.
Machine-readable output follows:
{“capabilities”: [“networks-management”, “static-ipaddresses”, “ipv6-deployment-ubuntu”, “devices-management”, “storage-deployment-ubuntu”, “network-deployment-ubuntu”, “bridging-interface-ubuntu”, “bridging-automatic-ubuntu”, “authenticate-api”], “version”: “3.3.2”, “subversion”: “”}

8

Still not able to reproduce with 3.2. There might be a bug somewhere.
Could you share your setup? Are you running multiple rack controllers? Could you check the logs under /var/snap/maas/common/log/rackd.log? Do you have a proxy?

9

We have 1 region/rack controller and 1 rack controller. We are not running any proxy. I see the root.tgz file under this directory on maas.

root@maas-1:/var/snap/maas/common/maas/boot-resources/snapshot-20230509-190527/centos/amd64/generic/9-custom/uploaded# ls
root-tgz

Below is the portion of the log that exhibits the error that I am seeing on boot. Let me try to downgrade to Maas 3.2 and see if the error goes away or not.

Thanks
John

2023-05-11 20:00:13 Uninitialized: [info] ClusterClient connection established (HOST:IPv6Address(type=‘TCP’, host=’::ffff:10.16.102.47’, port=45432, flowInfo=0, scopeID=0) PEER:IPv6Address(type=‘TCP’, host=’::ffff:10.16.102.47’, port=5252, flowInfo=0, scopeID=0))
2023-05-11 20:00:13 provisioningserver.rpc.clusterservice: [info] Event-loop ‘maas-1:pid=819465’ authenticated.
2023-05-11 20:00:13 provisioningserver.rpc.clusterservice: [info] Event-loop ‘maas-1:pid=819465’ authenticated.
2023-05-11 20:00:13 provisioningserver.rpc.clusterservice: [info] Rack controller ‘p7hxay’ registered (via maas-1:pid=819465) with MAAS version 3.3.3-13184-g.3e9972c19.
2023-05-11 20:00:13 provisioningserver.rpc.clusterservice: [info] Rack controller ‘p7hxay’ registered (via maas-1:pid=819465) with MAAS version 3.3.3-13184-g.3e9972c19.
2023-05-11 20:00:43 provisioningserver.rackdservices.dhcp_probe_service: [info] Probe for external DHCP servers started on interfaces: ens192.
2023-05-11 20:00:53 provisioningserver.rackdservices.dhcp_probe_service: [info] External DHCP probe complete.
2023-05-11 20:05:19 provisioningserver.rackdservices.tftp: [info] lpxelinux.0 requested by 10.16.99.217
2023-05-11 20:05:19 provisioningserver.rackdservices.tftp: [info] lpxelinux.0 requested by 10.16.99.217
2023-05-11 20:05:20 provisioningserver.rackdservices.http: [info] ldlinux.c32 requested by 10.16.99.217
2023-05-11 20:05:20 provisioningserver.rackdservices.http: [info] pxelinux.cfg/44454c4c-4d00-1058-8030-b7c04f5a5233 requested by 10.16.99.217
2023-05-11 20:05:20 provisioningserver.rackdservices.http: [info] pxelinux.cfg/01-b4-96-91-f8-03-50 requested by 10.16.99.217
2023-05-11 20:05:20 provisioningserver.rackdservices.http: [info] /images/ubuntu/amd64/ga-20.04/focal/stable/boot-kernel requested by 10.16.99.217
2023-05-11 20:05:20 provisioningserver.rackdservices.http: [info] /images/ubuntu/amd64/ga-20.04/focal/stable/boot-initrd requested by 10.16.99.217
2023-05-11 20:05:29 provisioningserver.rackdservices.http: [info] /images/ubuntu/amd64/ga-20.04/focal/stable/squashfs requested by 10.16.99.217
2023-05-11 20:06:17 provisioningserver.rackdservices.http: [info] /images/centos/amd64/generic/9-custom/uploaded/root-tgz requested by 10.16.99.217
2023-05-11 20:09:56 provisioningserver.rackdservices.tftp: [info] lpxelinux.0 requested by 10.16.99.217
2023-05-11 20:09:56 provisioningserver.rackdservices.tftp: [info] lpxelinux.0 requested by 10.16.99.217
2023-05-11 20:09:56 provisioningserver.rackdservices.http: [info] ldlinux.c32 requested by 10.16.99.217
2023-05-11 20:09:56 provisioningserver.rackdservices.http: [info] pxelinux.cfg/44454c4c-4d00-1058-8030-b7c04f5a5233 requested by 10.16.99.217
2023-05-11 20:09:56 provisioningserver.rackdservices.http: [info] pxelinux.cfg/01-b4-96-91-f8-03-50 requested by 10.16.99.217
2023-05-11 20:09:56 provisioningserver.rackdservices.http: [info] /images/centos/amd64/ga-20.04/9-custom/no-such-image/boot-kernel requested by 10.16.99.217
2023-05-11 20:10:43 provisioningserver.rackdservices.dhcp_probe_service: [info] Probe for external DHCP servers started on interfaces: ens192.
2023-05-11 20:10:43 sstreams: [info] maas:v2:download/maas:boot:centos:amd64:generic:8: to_add=[‘20211104_01’] to_remove=[]
2023-05-11 20:10:43 sstreams: [info] maas:v2:download/maas:boot:centos:amd64:generic:9-custom: to_add=[‘20230509’] to_remove=[]
2023-05-11 20:10:43 sstreams: [info] maas:v2:download/maas:boot:centos:amd64:generic:centos70: to_add=[‘20211103_01’] to_remove=[]
2023-05-11 20:10:43 sstreams: [info] maas:v2:download/maas:boot:custom:amd64:generic:rocky9.1: to_add=[‘20230505’] to_remove=[]

10

btw I found a very old bug which is very similar to your current situation https://bugs.launchpad.net/maas/+bug/1612203.

Could you paste here the output of maas admin boot-resources read? Gut feeling is that there is something wrong with the architecture type of the image as internally there is some logic that can end up setting that no-such-image

11

Here is the output below.

Thanks

root@maas-1:~# maas admin boot-resources read
Success.
Machine-readable output follows:
[
    {
        "id": 91,
        "type": "Synced",
        "name": "centos/8",
        "architecture": "amd64/generic",
        "resource_uri": "/MAAS/api/2.0/boot-resources/91/",
        "subarches": "generic"
    },
    {
        "id": 90,
        "type": "Synced",
        "name": "centos/centos70",
        "architecture": "amd64/generic",
        "resource_uri": "/MAAS/api/2.0/boot-resources/90/",
        "subarches": "generic"
    },
    {
        "id": 7,
        "type": "Synced",
        "name": "grub-efi-signed/uefi",
        "architecture": "amd64/generic",
        "resource_uri": "/MAAS/api/2.0/boot-resources/7/"
    },
    {
        "id": 8,
        "type": "Synced",
        "name": "grub-efi/uefi",
        "architecture": "arm64/generic",
        "resource_uri": "/MAAS/api/2.0/boot-resources/8/"
    },
    {
        "id": 9,
        "type": "Synced",
        "name": "grub-ieee1275/open-firmware",
        "architecture": "ppc64el/generic",
        "resource_uri": "/MAAS/api/2.0/boot-resources/9/"
    },
    {
        "id": 10,
        "type": "Synced",
        "name": "pxelinux/pxe",
        "architecture": "i386/generic",
        "resource_uri": "/MAAS/api/2.0/boot-resources/10/"
    },
    {
        "id": 1,
        "type": "Synced",
        "name": "ubuntu/focal",
        "architecture": "amd64/ga-20.04",
        "resource_uri": "/MAAS/api/2.0/boot-resources/1/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,ga-18.10,ga-19.04,ga-19.10,ga-20.04"
    },
    {
        "id": 2,
        "type": "Synced",
        "name": "ubuntu/focal",
        "architecture": "amd64/ga-20.04-lowlatency",
        "resource_uri": "/MAAS/api/2.0/boot-resources/2/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,ga-18.10,ga-19.04,ga-19.10,ga-20.04"
    },
    {
        "id": 3,
        "type": "Synced",
        "name": "ubuntu/focal",
        "architecture": "amd64/hwe-20.04",
        "resource_uri": "/MAAS/api/2.0/boot-resources/3/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,ga-18.10,ga-19.04,ga-19.10,ga-20.04"
    },
    {
        "id": 4,
        "type": "Synced",
        "name": "ubuntu/focal",
        "architecture": "amd64/hwe-20.04-edge",
        "resource_uri": "/MAAS/api/2.0/boot-resources/4/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,hwe-18.10,hwe-19.04,hwe-19.10,hwe-20.04"
    },
    {
        "id": 5,
        "type": "Synced",
        "name": "ubuntu/focal",
        "architecture": "amd64/hwe-20.04-lowlatency",
        "resource_uri": "/MAAS/api/2.0/boot-resources/5/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,ga-18.10,ga-19.04,ga-19.10,ga-20.04"
    },
    {
        "id": 6,
        "type": "Synced",
        "name": "ubuntu/focal",
        "architecture": "amd64/hwe-20.04-lowlatency-edge",
        "resource_uri": "/MAAS/api/2.0/boot-resources/6/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,hwe-18.10,hwe-19.04,hwe-19.10,hwe-20.04"
    },
    {
        "id": 49,
        "type": "Synced",
        "name": "ubuntu/jammy",
        "architecture": "amd64/ga-22.04",
        "resource_uri": "/MAAS/api/2.0/boot-resources/49/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,ga-18.10,ga-19.04,ga-19.10,ga-20.04,ga-20.10,ga-21.04,ga-21.10,ga-22.04"
    },
    {
        "id": 50,
        "type": "Synced",
        "name": "ubuntu/jammy",
        "architecture": "amd64/ga-22.04-lowlatency",
        "resource_uri": "/MAAS/api/2.0/boot-resources/50/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,ga-18.10,ga-19.04,ga-19.10,ga-20.04,ga-20.10,ga-21.04,ga-21.10,ga-22.04"
    },
    {
        "id": 85,
        "type": "Synced",
        "name": "ubuntu/jammy",
        "architecture": "amd64/hwe-22.04",
        "resource_uri": "/MAAS/api/2.0/boot-resources/85/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,ga-18.10,ga-19.04,ga-19.10,ga-20.04,ga-20.10,ga-21.04,ga-21.10,ga-22.04"
    },
    {
        "id": 86,
        "type": "Synced",
        "name": "ubuntu/jammy",
        "architecture": "amd64/hwe-22.04-edge",
        "resource_uri": "/MAAS/api/2.0/boot-resources/86/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,hwe-18.10,hwe-19.04,hwe-19.10,hwe-20.04,hwe-22.04"
    },
    {
        "id": 87,
        "type": "Synced",
        "name": "ubuntu/jammy",
        "architecture": "amd64/hwe-22.04-lowlatency",
        "resource_uri": "/MAAS/api/2.0/boot-resources/87/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,ga-18.10,ga-19.04,ga-19.10,ga-20.04,ga-20.10,ga-21.04,ga-21.10,ga-22.04"
    },
    {
        "id": 88,
        "type": "Synced",
        "name": "ubuntu/jammy",
        "architecture": "amd64/hwe-22.04-lowlatency-edge",
        "resource_uri": "/MAAS/api/2.0/boot-resources/88/",
        "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w,ga-16.04,ga-16.10,ga-17.04,ga-17.10,ga-18.04,hwe-18.10,hwe-19.04,hwe-19.10,hwe-20.04,hwe-22.04"
    },
    {
        "id": 97,
        "type": "Uploaded",
        "name": "centos/9-custom",
        "architecture": "amd64/generic",
        "resource_uri": "/MAAS/api/2.0/boot-resources/97/",
        "subarches": "generic",
        "title": "CentOS 9 Custom"
    },
    {
        "id": 92,
        "type": "Uploaded",
        "name": "esxi/6.7",
        "architecture": "amd64/generic",
        "resource_uri": "/MAAS/api/2.0/boot-resources/92/",
        "subarches": "generic",
        "title": "VMware ESXi 7.0"
    },
    {
        "id": 96,
        "type": "Uploaded",
        "name": "rocky9.1",
        "architecture": "amd64/generic",
        "resource_uri": "/MAAS/api/2.0/boot-resources/96/",
        "subarches": "generic",
        "title": "Rocky 9 Custom"
    }
]
12

Thanks. Could you now run maas admin rack-controller list-boot-images p7hxay? If you re-installed maas you might have to change p7hxay with the id you extract from maas admin rack-controllers read | grep system_id

13

Here’s the output.

Thanks

–
root@maas-1:~# maas admin rack-controller list-boot-images p7hxay
Success.
Machine-readable output follows:
{
“images”: [
{
“name”: “centos/8”,
“architecture”: “amd64”,
“subarches”: [
“generic”
]
},
{
“name”: “centos/centos70”,
“architecture”: “amd64”,
“subarches”: [
“generic”
]
},
{
“name”: “centos/9-custom”,
“architecture”: “amd64”,
“subarches”: [
“generic”
]
},
{
“name”: “esxi/6.7”,
“architecture”: “amd64”,
“subarches”: [
“generic”
]
},
{
“name”: “rocky9.1”,
“architecture”: “amd64”,
“subarches”: [
“generic”
]
},
{
“name”: “ubuntu/focal”,
“architecture”: “amd64”,
“subarches”: [
“ga-20.04”,
“ga-20.04-lowlatency”,
“generic”,
“hwe-20.04”,
“hwe-20.04-edge”,
“hwe-20.04-lowlatency”,
“hwe-20.04-lowlatency-edge”
]
},
{
“name”: “ubuntu/jammy”,
“architecture”: “amd64”,
“subarches”: [
“ga-22.04”,
“ga-22.04-lowlatency”,
“generic”,
“hwe-22.04”,
“hwe-22.04-edge”,
“hwe-22.04-lowlatency”,
“hwe-22.04-lowlatency-edge”
]
},
{
“name”: “bootloader/open-firmware”,
“architecture”: “ppc64el”,
“subarches”: [
“generic”
]
},
{
“name”: “bootloader/uefi”,
“architecture”: “amd64”,
“subarches”: [
“generic”
]
},
{
“name”: “bootloader/uefi”,
“architecture”: “arm64”,
“subarches”: [
“generic”
]
},
{
“name”: “bootloader/pxe”,
“architecture”: “i386”,
“subarches”: [
“generic”
]
}
],
“connected”: true,
“status”: “synced”
}

14

Thanks. Could you also

  1. try to deploy the centos9 image to one node - let’s say <machine_name>.
  2. run maas admin nodes read | jq '.[] | select(.fqdn == "<machine_name>.<domain>")' (you can find the machine_name and the domain in the ui - it’s just the name of the node that you see in the machine list page) and paste the result somewhere (like https://pastebin.ubuntu.com/)? In case you have confidential info there, remember to redact them.
15

Here is the output on pastebin.

Thanks

16

I’m finally able to reproduce this. I’ll keep you posted in the next days as soon as I have a bit of time to investigate more

1 Like
17

nice one, @r00ta! :slight_smile:

18

I am not seeing this issue in Maas 3.4 beta. I was able to deploy a Centos 9 Stream box.

Thanks

19

Hi @johnliaotvc, yep, I was about to inform you that 3.4 works fine.

If you face any other issue please let us know!

20

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.