Commissioning scripts with custom images

Users
1

Hello!
I created a custom ubuntu image with packer, but my script that works for the custom partition does not work with the custom image, more precisely the image automatically defines a disk capacity of its own. I set the “disk_size” parameter in the “ubuntu-lvm.pkr.hcl” file to 100G and then the generated script creates a /root partition of about 54GiB.

Finally, what I want to do is to use the 42-custom-lvm.sh commissioning script that works with the MAAS ubuntu standard image for the image I created with the packer. Which path should I follow for this.
Thanks.

My custom image configs:

ubuntu-lvm.pkr.hcl (I changed just disk_size)
scripts/curtin.sh (I added just “apt-get install ubuntu-desktop -yy”)

source "qemu" "lvm" {
  boot_command    = ["<wait>e<wait5>", "<down><wait><down><wait><down><wait2><end><wait5>", "<bs><bs><bs><bs><wait>autoinstall ---<wait><f10>"]
  boot_wait       = "2s"
  cpus            = 2
  disk_size       = "100G"
  format          = "raw"
  headless        = var.headless
  http_directory  = var.http_directory
  iso_checksum    = "file:http://releases.ubuntu.com/jammy/SHA256SUMS"
  iso_target_path = "packer_cache/ubuntu.iso"
  iso_url         = "https://releases.ubuntu.com/jammy/ubuntu-22.04.3-live-server-amd64.iso"
  memory          = 2048
  qemuargs = [
    ["-vga", "qxl"],
    ["-device", "virtio-blk-pci,drive=drive0,bootindex=0"],
    ["-device", "virtio-blk-pci,drive=cdrom0,bootindex=1"],
    ["-device", "virtio-blk-pci,drive=drive1,bootindex=2"],
    ["-drive", "if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF_CODE.fd"],
    ["-drive", "if=pflash,format=raw,file=OVMF_VARS.fd"],
    ["-drive", "file=output-lvm/packer-lvm,if=none,id=drive0,cache=writeback,discard=ignore,format=raw"],
    ["-drive", "file=seeds-lvm.iso,format=raw,cache=none,if=none,id=drive1,readonly=on"],
    ["-drive", "file=packer_cache/ubuntu.iso,if=none,id=cdrom0,media=cdrom"]
  ]
  shutdown_command       = "sudo -S shutdown -P now"
  ssh_handshake_attempts = 500
  ssh_password           = var.ssh_ubuntu_password
  ssh_timeout            = "5m"
  ssh_username           = "ubuntu"
  ssh_wait_timeout       = "5m"
}

build {
  sources = ["source.qemu.lvm"]

  provisioner "file" {
    destination = "/tmp/curtin-hooks"
    source      = "${path.root}/scripts/curtin-hooks"
  }

  provisioner "shell" {
    environment_vars  = ["HOME_DIR=/home/ubuntu", "http_proxy=${var.http_proxy}", "https_proxy=${var.https_proxy}", "no_proxy=${var.no_proxy}"]
    execute_command   = "echo 'ubuntu' | {{ .Vars }} sudo -S -E sh -eux '{{ .Path }}'"
    expect_disconnect = true
    scripts           = ["${path.root}/scripts/curtin.sh","${path.root}/scripts/networking.sh", "${path.root}/scripts/cleanup.sh"]
  }

  post-processor "compress" {
    output = "custom-ubuntu-lvm.dd.gz"
  }
}
2

Hi @elperro
what is 42-custom-lvm.sh?

3

This one:

4

Later in the thread the author said that the script was failing in some cases. Maybe @bedfordc do you have an updated version of that script?

5

Yes, I changed it a little bit. It works with ubuntu official image , but not with a custom image created by Packer. Now I’m trying to solve this custom partitioning problem, during the day I deep dive into the documentation but still, I have nothing.

Here is the updated one:

```
#!/usr/bin/env python3
####
#
#  Program: 42-custom-disk-layout.sh
#
#  Description:
#  This python script is to specify how to select and partition a disk in MAAS.
#  Inputs: the MAAS_RESOURCES_FILE from previous default scripts is a json
# construct detailing the hardware that is currently being "commissioned".
#  Outputs: the MAAS_STORAGE_CONFIG_FILE should contain a json structure
# that defines the selected hard disk and where to slice it up to the standard
# mountpoints for the OS installation, however that step continually failed.
# Instead this script loads all the data from the MAAS_RESOURCES_FILE, then
# adds the ['storage-extra'] block with the selected disk, partitions, and
# logical volumes.
#
#  Programmer:	Charles Bedford
#  History:
#    2022-09-30 - CHB - Initial revision
#
# --- Start MAAS 1.0 script metadata ---
# name: 42-custom-disk-layout
# title: Set layout for DELL Virtual disks
# description: Set layout for DELL Virtual disks
# script_type: commissioning
# timeout: 60
# --- End MAAS 1.0 script metadata ---
#
####
import json
import os
import sys

# Function definitions
def read_json_file(path):
    try:
        with open(path) as fd:
            return json.load(fd)
    except OSError as e:
        sys.exit(f"Failed to read {path}: {e}")
    except json.JSONDecodeError as e:
        sys.exit(f"Failed to parse {path}: {e}")

# Load the hardware from the json in the MAAS_RESOURCES_FILE
hardware = read_json_file(os.environ['MAAS_RESOURCES_FILE'])

####
#
#  This is the primary datastructure (in json format) with
# placeholders which are easily found and replaced below (DISK & MAX)
#
####
cfg = '''{
    "layout": {
        "DISK": {
            "type": "disk",
            "ptable": "gpt",
            "boot": true,
            "partitions": [
                { "name": "DISK1", "fs": "fat32", "size": "2G", "bootable": true },
                { "name": "DISK2", "size": "MAX" }
            ]
        },
        "vg": {
            "type": "lvm",
            "members": [ "DISK2" ],
            "volumes": [
                { "name": "lv1", "size": "100G", "fs": "ext4" },
                { "name": "lv2", "size": "16G", "fs": "swap" },
                { "name": "lv3", "size": "220G", "fs": "ext4" }
               
            ]
        }
    },
    "mounts": {
        "/": { "device": "lv1" },
        "/boot/efi": { "device": "DISK1" }, 
        "none": { "device": "lv2" },
        "/data": { "device": "lv3" }
        
    }
}'''

disks = hardware['resources']['storage']['disks']

# Initialize a few variables so we have consistent values below
primary = ''
primaryId = 0
diskSize = 0
index=0

# Iterate through all the disks defined for this machine
for disk in disks:
    # skip virtual mounted drives from maas
    if 'Virtual' in disk["model"] or 'Virtual' in disk['device_id']:
        continue

    # find the PERC or DELLBOSS id to use...
    if 'PERC' in disk["model"] or 'DELLBOSS' in disk["model"]:
            primary = disk['id']
            primaryId = index
            diskSize = disk['size']

    index+=1

# if the loop didn't find a PERC or DELLBOSS drive to use
# barring that it will have to be #0
if primary == '':
    primary=disks[0]['id']
    primaryId = 0
    diskSize = disks[0]['size']

# The disk ID (sda or whatever) that we want to use for booting...
diskId = hardware["resources"]["storage"]["disks"][primaryId]["id"]

# Load layoutDetail from the above structure
layoutDetail = json.loads(cfg)

# copy the DISK temporary structure over the top of the new diskId name
layoutDetail["layout"][diskId] = layoutDetail["layout"]["DISK"]

# Now delete the DISK temporary structure
del layoutDetail["layout"]["DISK"]

####
#
#  Cleanup the structure details
#
####
# set the names and sizes on the partitions:
# Note: the sizes are NOT based on 1024, but 1000.
# subtract the size of the fat32 partition and a constant...
mySize = ( diskSize - 2000000000 ) / 1000000000
num = 1
for p in layoutDetail["layout"][diskId]["partitions"]:
	p["name"] = diskId + str(num)
	num += 1;
	if p["size"] == "MAX":
		p["size"] = str(int(mySize)) + "G"
		savedP = p

# The Members of the lvm
layoutDetail["layout"]["vg"]["members"] = [ savedP["name"] ]

# last but not least - fix the one mount detail
layoutDetail["mounts"]["/boot/efi"]["device"] = diskId + "1"

####
#
#  Output
#
####
hardware["storage-extra"] = layoutDetail

print('Saving custom storage layout to ' + os.environ['MAAS_RESOURCES_FILE'])
print(json.dumps(hardware))

with open(os.environ['MAAS_RESOURCES_FILE'], 'w') as fd:
    json.dump(hardware, fd)
```
6

I am going crazy. I created a custom ubuntu 2204 image using packer templates (ubuntu-lvm.pkr.hcl) as described in the documentation. The only change I made to this image was to add the line “apt-get install ubuntu-desktop -yy” in “scripts/curtin.sh”. At the end of this process, the image created works properly, there are no problems in the deployment process.

It creates as much disk space as the size in the "disk_size = “8G” parameter in the hcl file, the rest of the disk remains unallocated.

1- How can I create a partition image that I want only for this custom image?

There is a section like this in the documentation but no other details:

About configuring deployed machine storage

If you deploy a machine with a custom Ubuntu image, you will also want to be able to configure storage, just like you would do with any other machine. MAAS facilitates changes to the storage configuration. You can resize the root partition, as well as attaching and formatting any additional block devices you may desire.

7

could you summarize what is the layout you’d like to apply to the machines?

8

/ 100 Gib
/tmp 10 Gib
/data (rest of the space, I create 10Gib with custom storage commissioning script then extend it with cloud-init)

9

I haven’t yet set the partition schema I want for a custom created ubuntu image with packer during installation withMAAS. The temporary solution I found, for now, is a few manipulations with cloud-init.
Also the “custom storage” (42-custom-disk-layout.sh) script works well with “rhel” custom image that I created also with packer, it creates the desired partition table without any problems, but for some reason, it does not work with Ubuntu.

runcmd:
  - mkdir /data
  - echo -e "n\nn\n4\n\n\nw\nq" | fdisk /dev/sda
  - sleep 2
  - pvcreate /dev/sda4 
  - sleep 2
  - vgextend os-vg /dev/sda4 
  - sleep 2
  - lvextend -l +100%FREE /dev/os-vg/os-lv 
  - sleep 2
  - resize2fs /dev/os-vg/os-lv
10

Hey there, @elperro,

This one’s been sitting here a while with no resolution, so I figure some spitballing by a technical author couldn’t make it much worse. Here’s what comes to my (very strange) mind:

  • For Ubuntu, the curtin installer that MAAS uses by default expects a certain partition layout that may conflict with custom storage configs. Using a preseed file instead with curtin may help define the partitions.
  • Try creating a custom curtin storage config file like 40-custom-storage.yaml that specifies the desired layout, and pass that to MAAS on import or with curtin preseeds.
  • Use a late command in curtin preseeds to resize partitions after install, rather than trying to predefine sizes which may get overwritten.
  • Consider switching to using cloud-init for provisioning instead, where you have more control over disk configuration in user-data scripts.
  • The ubuntu-lvm Packer template is designed for LVM by default, so adapting that to partitioned disks rather than LVM volumes could be tricky.
  • For testing, try deploying the image first without any custom storage to validate the base install, then work on storage configuration separately.
  • Check the curtin and cloud-init logs on the deployed nodes for any errors applying custom storage configs.

Obviously, the key is prolly getting curtin and cloud-init working together with your desired layouts.