Custom storage layouts using preseeds

I was looking for a way to implement a custom storage configuration for a class of machines. In my case, I wanted MAAS to detect all disks of a given model and set them up in a RAID array, and then choose a disk of a different model for boot/root. I could of course do this in the UI - but that would need to be done for each system, and would be blown away when a system was recommissioned. I wanted something more dynamic. And ideally something I could maintain in git.

My first thought was to do this as a static preseed. I found this blog where the author provided a preseed to override curtin‘s partitioning_commands: YAML. That mostly worked - but I quickly discovered some limitations. See, MAAS has already instantiated a full internal model of storage and processed that down into curtin configuration. If you override part of that processed config, you risk corrupting that model. In my case things fell over with MAAS wanting to use a specific disk for the bootloader install which didn’t match how I was partitioning things behind its back. No, I really needed to modify MAAS’ model of the storage so it retained a holistic view of the system.

A colleague of mine had solved a similar problem and showed me how they had solved it. Preseed files permit inline Python code that gets processed at deploy time. Using this interface you can manipulate internal MAAS objects, such as the storage config. So I can basically use this to implement my own storage layout logic!

During commissioning, MAAS has built up a set of objects representing the node and the various block devices it has. Then, based on the storage layout type selected in the UI, MAAS has created an additional set of “logical” objects to fit that layout type - these objects represent partitions, filesystems, etc. What I needed to do was clear out those abstract objects and create my own partitioning/filesystem objects on top of the physical disk objects MAAS knows about. Basically implementing my own layout type.

Here’s what it looks like:

#cloud-config
showtrace: true
verbosity: 3

{{py:
DGX_Platforms = {
    'DGXA100': {
        'RootDiskModel': 'SAMSUNG MZ1LB1T9HALS-00007',
        'RAIDDiskModel': 'SAMSUNG MZWLJ3T8HBLS-00007',
    },
    'NVIDIA DGX-2': {
        'RootDiskModel': 'SAMSUNG MZ1LW960HMJP-00003',
        'RAIDDiskModel': 'Micron_9200_MTFDHAL3T8TCT',
    },
}

# All systems use the same preseed, so we need to
# vary what we do depending on the server model
platform = node.get_metadata()['mainboard_product']
if platform in DGX_Platforms.keys():
    import random
    from maasserver.models import BlockDevice
    from maasserver.models import Filesystem
    from maasserver.storage_layouts import FlatStorageLayout
    from maasserver.models.filesystemgroup import RAIDManager
    from maasserver.enum import FILESYSTEM_GROUP_TYPE
    from maasserver.enum import FILESYSTEM_TYPE

    # <Boilerplate> 
    # Clear the layout MAAS generated (must be done before modifying
    # storage at this stage).
    node._clear_acquired_filesystems()
    # Clear partitions
    node._clear_full_storage_configuration()
    # </Boilerplate>

    root_candidates = []
    raid_candidates = []
    blockdevs = list(BlockDevice.objects.filter(node=node))
    for bdev in blockdevs:
        model = bdev.physicalblockdevice.model
        if model == DGX_Platforms[platform]['RootDiskModel']:
            root_candidates.append(bdev)
        elif model == DGX_Platforms[platform]['RAIDDiskModel']:
            raid_candidates.append(bdev)

    # Any of the possible root devices will work, just pick one
    node.boot_disk = random.choice(root_candidates).physicalblockdevice
    storage_layout = FlatStorageLayout(node)
    storage_layout.configure()
    raid_members = []
    for bdev in raid_candidates:
        raid_members.append(bdev.create_partition())
    r = RAIDManager()
    raiddev = r.create_raid(
        FILESYSTEM_GROUP_TYPE.RAID_0, partitions=raid_members
    )
    Filesystem.objects.create(
        block_device=raiddev.virtual_device,
        mount_point='/raid',
        fstype=FILESYSTEM_TYPE.EXT4,
    )
    
    # <Boilerplate> 
    # Force refresh the node from the database and clear the object relation
    # cache so that the following code uses the new storage layout.
    node.refresh_from_db()

    if hasattr(node, '_prefetched_objects_cache'):	
        delattr(node, '_prefetched_objects_cache')

    node._create_acquired_filesystems()
    # </Boilerplate>
}}

debconf_selections:
 maas: |
  {{for line in str(curtin_preseed).splitlines()}}
  {{line}}
  {{endfor}}

swap:
  size: 0

late_commands:
  maas: ['wget', '--no-proxy', '{{node_disable_pxe_url}}', '--post-data', '{{node_disable_pxe_data}}', '-O', '/dev/null']

What’s nice is that you can do most of the devel/debug without having to deploy the system. You can modify the preseed and then dump the curtin config it would generate using the MAAS CLI - maas <profile> machine get-curtin-config <system_id>. If you’re not sure what a “correct” curtin config would look like for your desired layout, you can manually create one using the MAAS UI’s Storage tab, and dump the resulting curtin config to use as a target.

If you want to poke around at the various MAAS storage objects interactively, you can get a python (or ipython if installed) shell in a similar environment by running sudo maas-region shell. (Note: I don’t have a good way of doing this in a snap-install - I used a deb install for debug). To get a handle on a node object, you can do something like:

$ sudo maas-region shell
Python 3.8.5 (default, Jul 28 2020, 12:59:40) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from maasserver.models import Machine

In [2]: machine = Machine.objects.get(hostname="myhostname")

In [3]: node = machine.as_node()

In [4]:
1 Like

sudo snap run --shell maas -c 'maas-region shell'

2 Likes