Streaming Custom Ubuntu Images

Discourse
1

Hello there,

I am working on a prototype in which I will have to stream custom Ubuntu images into MAAS.

The idea is use packer-maas as a means to build the custom image. Then upload these custom images to Sonatype Nexus. These will be triggered by a scheduled CI build. Finally update the MAAS UI Image configuration to use the Nexus URL as an Image source.

What is working so far:
Before building the CI scripts, I tried to make the images manually using packer-maas. This works and I can customize the cloud image and build it. After that, I hand-built the needed streams json files using the simplestreams readme files as reference.

I was then able to change the image source in the MAAS UI and make it accept my custom image source stream.

What is not working:
Although the MAAS UI does not report any errors on the image source change. I can see in the MAAS logs that the images are being ignored. Here are some log examples below:

2024-06-16T13:45:56.478202+00:00 playground maas.import-images: [warn] Ignoring unsupported product com.ubuntu.maas.candidate:ubuntu-bases:22.04:amd64
2024-06-16T13:45:56.478266+00:00 playground maas.import-images: [warn] Ignoring unsupported product com.ubuntu.maas.candidate:ubuntu-bases:22.04:amd64

What I tried to mitigate the issue I am encountering:
I did a search on how the streams are being built and stumbled upon the maas-images launchpad repository.

I cloned it and updated conf/packer-maas.yaml with configurations to build Ubuntu images.

I tried to create a stream with the following command:

sudo bin/meph2-import --force conf/packer-maas.yaml out.d

This command succeeds I am able to get a folder structure and the JSON files for the custom image. However, upon testing (reuploading them to my local nexus repository and setting up MAAS UI to use that as an Image source), I still get the aforementioned warnings that the images I have are being ignored.

Questions
I tried to find documentation on how to achieve this, however I did not find any that specifically talks about streaming custom images. As far as the docs go, all points to using the MAAS endpoint to upload the custom images images. Is streaming custom images achievable or the streams are limited to the ‘standard’ Ubuntu images?

The Ubuntu packer template builds a cloud image (tar.gz file) which to my limited reading of the maas-images source code, is just being renamed – no extra processing of the file. I tried without modifiying the template and still the built images are being ignored by MAAS. Do I need extra modifications on the packer template?

I hope someone has worked on something similar or has knowledge on how to properly build/prepare the images so that it can be accepted by MAAS as an image source. Thanks!

2

Adding some more information.

Here is how the folder structure looks like:

├── streams
│   └── v1
│       ├── com.ubuntu.maas:candidate:ubuntu-bases-download.json
│       └── index.json
└── ubuntu
    └── jammy
        └── amd64
            └── 20240616_01
                ├── root-tgz
                └── root-tgz.manifest

Here is how the JSON files look like:

{
 "format": "index:1.0",
 "index": {
  "com.ubuntu.maas:candidate:ubuntu-bases-download": {
   "datatype": "image-ids",
   "format": "products:1.0",
   "path": "streams/v1/com.ubuntu.maas:candidate:ubuntu-bases-download.json",
   "products": [
    "com.ubuntu.maas.candidate:ubuntu-bases:22.04:amd64"
   ],
   "updated": "Sun, 16 Jun 2024 14:41:38 +0000"
  }
 },
 "updated": "Sun, 16 Jun 2024 14:46:20 +0000"
}

products JSON:

{
 "content_id": "com.ubuntu.maas:candidate:ubuntu-bases-download",
 "datatype": "image-ids",
 "format": "products:1.0",
 "products": {
  "com.ubuntu.maas.candidate:ubuntu-bases:22.04:amd64": {
   "arch": "amd64",
   "label": "candidate",
   "os": "ubuntu",
   "release": "jammy",
   "release_title": "Jammy Jellyfish",
   "subarch": "generic",
   "subarches": "generic",
   "support_eol": "2030-05-01",
   "version": "22.04",
   "versions": {
    "20240616_01": {
     "items": {
      "manifest": {
       "ftype": "manifest",
       "path": "ubuntu/jammy/amd64/20240616_01/root-tgz.manifest",
       "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
       "size": 0
      },
      "root-tgz": {
       "ftype": "root-tgz",
       "path": "ubuntu/jammy/amd64/20240616_01/root-tgz",
       "sha256": "5f041313cedfdceb7ab07180523c5db0bc06d741c1169dc43ff63c911f1f3b3b",
       "size": 540216476
      }
     }
    }
   }
  }
 },
 "updated": "Sun, 16 Jun 2024 14:41:38 +0000"
}

The JSON files as well as the image files were generated via:

sudo bin/meph2-import --force conf/packer-maas.yaml out.d