Simplestreams mirror for custom images

Hi,

I would like to setup a local custom images mirror for MaaS but Simplestreams repositories are … well … not that simple to setup.
I tried to find documentation describing the process to build such setup + tutorials on how to build custom images with the right format for SimpleStreams but there are very few things out there.
I already have some build scripts for building custom images based on debootstrap/chroot/ansible, I like it that way and can do about anything with this but how do I go from my custom image to a complete simplestreams repo ?

Any help gladly welcomed.

Regards.

Canonical provides LP:simplestreams to help create a stream but I agree it can be difficult. The MAAS team uses LP:maas-images to generate the public stream at images.maas.io. I integrated support for packer-maas when adding CentOS 8 to the stream. Below is what we use to generate the CentOS stream.

#!/bin/bash -ex

# Packer only recognizes upper case proxy environment variables - https://github.com/hashicorp/packer/issues/2275
export HTTP_PROXY="$http_proxy"
export HTTPS_PROXY="$https_proxy"

# meph2-import will use PACKER_PATH from the environment variable when defined, otherwise it searches $PATH
#export PACKER_PATH=$WORKSPACE_TMP/packer

# Cache ISOs into WORKSPACE_TMP so the cache survives multiple builds.
export PACKER_CACHE_DIR=$WORKSPACE_TMP

# Disable checking for a new Packer version, it's blocked anyway.
export CHECKPOINT_DISABLE=1

# Needed to enable serial output
export PACKER_LOG=1

sudo -En $WORKSPACE/maas-images/bin/meph2-import -u $([ "$FORCE" == "true" ] && echo "-f") packer-maas.yaml $WORKSPACE/centos-stream
sudo -En chown -R jenkins:jenkins $WORKSPACE/centos-stream

@Hybrid512, i feel your pain. as the tech author, i sometimes find i need to document other people’s stuff (like jq, for example) because the native product is more complex. to that end, can you give me an idea of the architecture you’re wanting to support with mirrors (i.e., size, machine count, number of subnets, etc.) ? it may help me produce a better MAAS-flavored simplestreams tutorial if i know something about the use cases. tia.

Hi @billwear, many thanks for your attention.

Well here is my use case scenario.

I intend to setup a CI/CD based on Gitlab/Gitlab-CI to build custom cloud images for our platform.
These images will be built with a custom script based on debootstrap/chroot/ansible mostly.
Once built, these images shall be pushed to a SimpleStreams repository that will be used for both MaaS (as a custom replacement repo instead of the maas.io repo) and for Openstack with the help of the Glance SimpleStreams sync Charm.
This repo will host ubuntu images but not only, it will also host VMWare ESX images and RHEL/Centos images.

So, my question is : how do you setup such repo ?
All the tutorials I found (and there aren’t much to say the least) talk about mirroring an existing repo, not building your own except one I found for LXD which more or less was doing the same thing that I want to achieve.

I see how the repo si organized and most things are quite easily “reverse engineerable” but there are things like GPG signatures and json metadata that are not that trivial.

Can you help me on that ? Could be the basis for a good tutorial on one of the Canonical blogs I guess.

Best regards.