Using LXD projects with MAAS (snap/3.0/CLI)

LXD and MAAS are separate products, and it’s useful to allow them to interact as equals, covering a much wider range of use cases. To allow each of them to follow their own operational models, but still allow them to work together, we’ve taken advantage of LXD projects. This page provides a well-rounded look at projects – and how they’re used with MAAS – from four perspectives:

These four major sections should help you quickly get up to speed on the way that projects couple LXD and MAAS, and how to use this linkage to your advantage.

Can you explain how LXD projects work with MAAS?

It may be beneficial to understand how LXD projects fit into the overall MAAS ecosystem. LXD projects are not intended to be MAAS projects; they are only intended to limit which LXD containers and VMs are available to MAAS. Managing machines in projects or groups – within MAAS – is the function of resource pools. This section will help you understand the distinction and rationale behind our interaction with LXD projects in MAAS.

Four questions you may have about LXD projects, in general:

What are LXD projects?

LXD projects are a feature of the LXD lightweight container hypervisor – a next-generation container manager which makes containers as easy to manage as virtual machines. With LXD, you can create lots of containers, providing different services across many different use cases. As a result of this flexibility, it can become confusing to keep track of exactly which containers are providing what services to answer which use case.

To help with this potential confusion, LXD provides a “projects” feature, which allows you to group one or more containers together into related projects. These projects can be manipulated and managed with the same lxc tool used to manage the containers and virtual machines themselves.

Since MAAS makes use of LXD as a VM host, it’s useful to be able to manipulate and manage projects not only through MAAS, but also directly through LXD. In fact, to properly scope MAAS access to your LXD resources, it’s essential that you understand how to use LXD projects in some detail.

How did we get here?

Prior versions of MAAS implemented LXD VM hosts, but did so in a strongly-coupled way; that is, MAAS essentially took control of the VMs, disallowing or overriding some direct LXD controls. In a sense, MAAS became the owner, rather than simply a separate program, communicating interdependently with LXD to accomplish user purposes.

This was less than ideal. For example, MAAS would discover all running LXD VMs and immediately commission them, essentially performing a “clean install” and wiping out their contents. This behaviour isn’t suitable when MAAS is connected to a deployed LXD with VMs already running services that may not have been targeted for enlistment by MAAS.

With the release of MAAS 3.0 version, MAAS now becomes a LXD tenant, rather than an owner.

MAAS now controls VMs at a project level

Essentially, MAAS still commissions any VMs it discovers within a VM host, but the scope of the discovery is now limited to a single project. That project can contain any subset of the existing VMs, from zero to all of them. Any VM that’s already in a project assigned to MAAS will still be re-commissioned, which is the normal behaviour of MAAS upon network discovery.

Here are the criteria which drove this decision:

  • It should be possible for an administrator to select which project(s) MAAS manages, and thus which machines will be automatically enlisted and commissioned.
  • It should be possible for MAAS to create and self-assign a new (empty) project, so that the user can compose new VMs within LXD from within the MAAS interface.
  • No per-project features should be enabled by default in MAAS-created projects.
  • The project must be explicitly specified when creating the LXD VM host.
  • When a LXD VM host is added, only the existing VMs assigned to the selected project (if any) should be visible to MAAS (and thus, automatically commissioned), and VMs in non-MAAS projects are not affected in any way.
  • When a LXD VM host is deleted, the default behaviour should be for MAAS to leave existing VMs, rather than automatically deleting them; VMs in non-MAAS projects are not affected in any way.
  • MAAS will not create VMs in projects that it doesn’t own.
  • When a VM is composed in MAAS, it’s created in the target LXD project.

In addition, a MAAS administrator should be able to:

  • refresh information about a specific LXD VM host, and receive correct, up-to-date, and timely status.
  • explicitly specify connections between networks and VM interfaces.
  • deploy a machine directly as an LXD VM host, connected to MAAS from the outset.

These criteria were fully met in the MAAS LXD tenant implementation released as part of MAAS 3.0.

What are some alternatives to LXD projects in MAAS?

You can see from the discussion above that LXD projects were used primarily to cordon off some LXD VMs from MAAS, to avoid them from being enlisted and commissioned by MAAS (and thus, essentially, destroyed, from an operational perspective). These LXD project features provide some limited capabilities to group machines. Because LXD projects only deal with LXD VMs and VM hosts, they are not a complete or comprehensive set of project tools. MAAS already has those machine grouping capabilities in the form of resource pools.

We realised, though, as we were working with LXD projects, that we could vastly improve the documentation for resource pools in the area of projects and project management. You’ll find significant new material in the resource pools section of the doc. We also realised that it would be helpful to have “real-time tags” for machines, that is, annotations that persist only while a machine is allocated or deployed. These new “tags” are known as workload annotations, and they have also been given a thorough treatment, also with their own page in the left-hand navigation.

Why did we implement LXD projects with MAAS?

As you can infer from some of the discussion above, MAAS and LXD are separate products. It’s beneficial for them to interact, but not if one product cannot be controlled and managed independently of the other. The combination of MAAS and LXD can cover a much broader set of use cases between them if they act as independent tools. While the most loosely-coupled model would have MAAS selecting individual VMs, that model isn’t compatible with how MAAS works with other networks and devices, so we chose to implement projects as a way to section off LXD VMs for normal use by MAAS.

Can you teach me how to use LXD projects?

A good understanding of LXD projects is essential for those using LXD VM hosts, especially if you plan to include non-MAAS-controlled VMs in your LXD instance. Normally, we wouldn’t revisit instructions found elsewhere, but because the discussion flows quickly and naturally into MAAS-related usage, it seemed prudent to give a light overview of some basic feature information.

There are two avenues for learning about LXD projects:

These sections are arranged as tutorials, rather than simple, step-by-step how-to guides. If you’re looking for more concise procedures, see the best practices section later in this document.

How do I use LXD projects with LXD by itself?

Even without considering MAAS, there are a number of steps you can take to create and manage projects in LXD, directly from the command line. In this section, we’ll cover the following basic activities:

We won’t cover viewing and editing project configurations, and we won’t address theory or give overly detailed explanations. If you need any of that information, you should consult the LXD project documentation or the Ubuntu LXD documentation.

How do I list LXD projects?

Before you try to manipulate projects, it’s useful to understand how to list them, so that you can check your results as you go. If you’ve successfully installed and initialised lxd, you should be able to list projects. A basic project list can be obtained with the following command:

lxd project list

You should get a listing something like this:

+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |       DESCRIPTION       | USED BY |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| default (current)   | YES    | YES      | YES             | YES      | Default LXD project     | 7       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| pg_basebackup-tests | NO     | YES      | NO              | NO       | Project managed by MAAS | 16      |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+

Note that this particular instantiation of LXD has two projects: the default project (which generally always exists in LXD), and a project called pg_basebackup-tests which is already managed by MAAS.

There is a column labelled USED BY, which tabulates the number of entities contained within the project. There isn’t a project-related command to get a list of the containers and VMs within a project. Instead, you use the LXC command lxc list:

lxc list

which yields something like the following tabulated listing:

+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
|      NAME       |  STATE  | IPV4 |                    IPV6                     |      TYPE       | SNAPSHOTS |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| trusty-drake    | STOPPED |      |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| upward-stallion | RUNNING |      | fd42:ec:5a53:59d2:216:3eff:febf:7fa7 (eth0) | CONTAINER       | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| witty-lizard    | STOPPED |      |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+

How do you know which project you’re listing? The most reliable way is to first list projects and see which one is marked (current), like this:

lxc project list

As you see in the sample output, the currently visible and accessible project is listed as (current) in the project listing:

+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |       DESCRIPTION       | USED BY |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| default (current)   | YES    | YES      | YES             | YES      | Default LXD project     | 7       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| pg_basebackup-tests | NO     | YES      | NO              | NO       | Project managed by MAAS | 16      |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+

We’ll show you how to switch to a different project further along in this tutorial.

How do I create a LXD project?

Suppose that you’re about to create a MAAS VM host, and you want a specific project named “maas-vm-host-1” for this particular situation. You can create that project with the following command:

$ lxc project create maas-vm-host-1
Project maas-vm-host-1 created

When you check your work with lxc project list, you’ll see that LXD did not automatically switch to the new project:

+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |       DESCRIPTION       | USED BY |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| default (current)   | YES    | YES      | YES             | YES      | Default LXD project     | 7       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| maas-vm-host-1      | YES    | YES      | YES             | NO       |                         | 1       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| pg_basebackup-tests | NO     | YES      | NO              | NO       | Project managed by MAAS | 16      |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+

The lxc tool generally does only what you ask, nothing more.

How do I delete a LXD project?

Now, suppose that you decide you don’t need this project yet. No worries: you can easily delete it like this:

$ lxc project delete maas-vm-host-1
Project maas-vm-host-1 deleted

You can check that it was successfully deleted with the lxc project list command:

+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |       DESCRIPTION       | USED BY |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| default (current)   | YES    | YES      | YES             | YES      | Default LXD project     | 7       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| pg_basebackup-tests | NO     | YES      | NO              | NO       | Project managed by MAAS | 16      |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+

How do I rename a LXD project?

On the other hand, maybe you didn’t need to actually delete that project, just change the name to maas-vm-host-001, which is what you really wanted in the first place. Consider your original project name, maas-vm-host-1:

+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |       DESCRIPTION       | USED BY |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| default (current)   | YES    | YES      | YES             | YES      | Default LXD project     | 7       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| maas-vm-host-1      | YES    | YES      | YES             | NO       |                         | 1       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| pg_basebackup-tests | NO     | YES      | NO              | NO       | Project managed by MAAS | 16      |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+

You can quickly and easily change the project name like this:

$ lxc project rename maas-vm-host-1 maas-vm-host-001
Project maas-vm-host-1 renamed to maas-vm-host-001
$ lxc project list
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |       DESCRIPTION       | USED BY |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| default (current)   | YES    | YES      | YES             | YES      | Default LXD project     | 7       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| maas-vm-host-001    | YES    | YES      | YES             | NO       |                         | 1       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| pg_basebackup-tests | NO     | YES      | NO              | NO       | Project managed by MAAS | 16      |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+

From now on, we’ll be combining command output with the command invocation, most of the time.

How do I switch LXD projects?

You can choose which LXD project is currently visible and accessible, that is, you can choose which project will be acted on by most of the other project commands:

$ lxc project list
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |       DESCRIPTION       | USED BY |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| default (current)   | YES    | YES      | YES             | YES      | Default LXD project     | 7       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| maas-vm-host-001    | YES    | YES      | YES             | NO       |                         | 1       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| pg_basebackup-tests | NO     | YES      | NO              | NO       | Project managed by MAAS | 16      |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+

Only the project marked (current) in the project listing can be manipulated, for the most part, with the obvious exceptions of command that take project names (like “create,” “delete,” and so forth). For example, using lxc list to enumerate the names of containers and VMs limits its scope to the current project:

$ lxc list
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
|      NAME       |  STATE  | IPV4 |                    IPV6                     |      TYPE       | SNAPSHOTS |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| trusty-drake    | STOPPED |      |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| upward-stallion | RUNNING |      | fd42:ec:5a53:59d2:216:3eff:febf:7fa7 (eth0) | CONTAINER       | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| witty-lizard    | STOPPED |      |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+

Suppose I want to know what all those “USE BY” things are in that pg_basebackup-tests project? Well, the most straightforward way to get that list is to first switch projects, then repeat the list command, like this:

$ lxc list
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
|      NAME       |  STATE  | IPV4 |                    IPV6                     |      TYPE       | SNAPSHOTS |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| trusty-drake    | STOPPED |      |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| upward-stallion | RUNNING |      | fd42:ec:5a53:59d2:216:3eff:febf:7fa7 (eth0) | CONTAINER       | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| witty-lizard    | STOPPED |      |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+

You can see in the above listing that we’ve switched to the “…-tests” project. Now when we do a container list, we’ll see a different set:

$ lxc list
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
|      NAME       |  STATE  | IPV4 |                    IPV6                     |      TYPE       | SNAPSHOTS |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| trusty-drake    | STOPPED |      |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| upward-stallion | RUNNING |      | fd42:ec:5a53:59d2:216:3eff:febf:7fa7 (eth0) | CONTAINER       | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+
| witty-lizard    | STOPPED |      |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+---------------------------------------------+-----------------+-----------+

It’s good practice to always switch projects carefully, so you’re not operating in some other project and creating chaos by accident.

How do I get a summary of LXD project resources?

We said that lxc commands operate on the current project most of the time. We gave that caveat because of commands like lxc project info, which requires a project name to get any usable output. For example, if you just type lxc project info, you’ll just get some “help” output:

$ lxc project info
Description:
  Get a summary of resource allocations

Usage:
  lxc project info [<remote>:]<project> <key> [flags]

Flags:
      --format   Format (csv|json|table|yaml) (default "table")

Global Flags:
      --debug            Show all debug messages
      --force-local      Force using the local unix socket
  -h, --help             Print help
      --project string   Override the source project
  -q, --quiet            Don't show progress information
  -v, --verbose          Show all information messages
      --version          Print version number

You can see from the help listing that a project name is required. Let’s try that again with a fairly large project:

$ lxc project info pg_basebackup-tests
+------------------+-----------+----------+
|     RESOURCE     |   LIMIT   |  USAGE   |
+------------------+-----------+----------+
| CONTAINERS       | UNLIMITED | 0        |
+------------------+-----------+----------+
| CPU              | UNLIMITED | 15       |
+------------------+-----------+----------+
| DISK             | UNLIMITED | 120.00GB |
+------------------+-----------+----------+
| INSTANCES        | UNLIMITED | 15       |
+------------------+-----------+----------+
| MEMORY           | UNLIMITED | 32.21GB  |
+------------------+-----------+----------+
| NETWORKS         | UNLIMITED | 0        |
+------------------+-----------+----------+
| PROCESSES        | UNLIMITED | 0        |
+------------------+-----------+----------+
| VIRTUAL-MACHINES | UNLIMITED | 15       |
+------------------+-----------+----------+

Here we see that the pg_basebackup-tests file has no containers, 15 VMs, 120GB of disk space used, etc. You can do this for any project, even if it’s not the current project, so from where we are here (in the pg_basebackup-tests project), we can still check resources in the default project:

+------------------+-----------+---------+
|     RESOURCE     |   LIMIT   |  USAGE  |
+------------------+-----------+---------+
| CONTAINERS       | UNLIMITED | 1       |
+------------------+-----------+---------+
| CPU              | UNLIMITED | 2       |
+------------------+-----------+---------+
| DISK             | UNLIMITED | 16.00GB |
+------------------+-----------+---------+
| INSTANCES        | UNLIMITED | 3       |
+------------------+-----------+---------+
| MEMORY           | UNLIMITED | 4.29GB  |
+------------------+-----------+---------+
| NETWORKS         | UNLIMITED | 2       |
+------------------+-----------+---------+
| PROCESSES        | UNLIMITED | 0       |
+------------------+-----------+---------+
| VIRTUAL-MACHINES | UNLIMITED | 2       |
+------------------+-----------+---------+

Note that lxc project info requires a project name. As mentioned earlier, typing the command without a project name just gives you a help message, not the stats for the default or current projects.

How do I show LXD project options?

You’ll remember that the “USED BY” column seemed to list more entities than there were containers or VMs. For example, the default project is “USED BY” seven entities, but only shows three containers or VMs:

$ lxc project list
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |       DESCRIPTION       | USED BY |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| default (current)   | YES    | YES      | YES             | YES      | Default LXD project     | 7       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| maas-vm-host-001    | YES    | YES      | YES             | NO       |                         | 1       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| pg_basebackup-tests | NO     | YES      | NO              | NO       | Project managed by MAAS | 16      |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+

You can make more sense of the “USED BY” column, and get a lot more information about your project, by using the lxc project show command:

$ lxc project show default
config:
  features.images: "true"
  features.networks: "true"
  features.profiles: "true"
  features.storage.volumes: "true"
description: Default LXD project
name: default
used_by:
- /1.0/images/9a30ffb2faeea61cce6012c63071a1f1504a76e1dbbe03e575cc313170fdaf43
- /1.0/instances/trusty-drake
- /1.0/instances/upward-stallion
- /1.0/instances/witty-lizard
- /1.0/networks/lxdbr0
- /1.0/networks/lxdbr1
- /1.0/profiles/default

Here you’ll see several categories of information, notably as list of entities that are using this project. For example, there are three VMs/containers, two networks, one image, and the default profile.

What’s really interesting, though, is that the pg_basebackup-tests project is only used by 16 entities – but there are 15 VMs in that project. What’s that discrepancy about? Well, we can find out by showing the options for that project:

$ lxc project show options pg_basebackup-tests
Description:
  Show project options

Usage:
  lxc project show [<remote>:]<project> [flags]

Global Flags:
      --debug            Show all debug messages
      --force-local      Force using the local unix socket
  -h, --help             Print help
      --project string   Override the source project
  -q, --quiet            Don't show progress information
  -v, --verbose          Show all information messages
      --version          Print version number
Error: Invalid number of arguments

Hmm, that’s weird. The project name is right, but it’s giving an error message. That’s because we used dashes in the project name, and the shell is trying to pick off everything after the first dash as an option to lxc project show maas. To get around this, we’d need to put the name in quotes, like this:

$ lxc project show "pg_basebackup-tests"
config:
  features.images: "false"
  features.profiles: "true"
  features.storage.volumes: "false"
description: Project managed by MAAS
name: pg_basebackup-tests
used_by:
- /1.0/instances/able-camel?project=pg_basebackup-tests
- /1.0/instances/alert-newt?project=pg_basebackup-tests
- /1.0/instances/casual-swan?project=pg_basebackup-tests
- /1.0/instances/exotic-grouse?project=pg_basebackup-tests
- /1.0/instances/flying-vervet?project=pg_basebackup-tests
- /1.0/instances/helped-thrush?project=pg_basebackup-tests
- /1.0/instances/main-mite?project=pg_basebackup-tests
- /1.0/instances/nearby-camel?project=pg_basebackup-tests
- /1.0/instances/noble-cod?project=pg_basebackup-tests
- /1.0/instances/poetic-parrot?project=pg_basebackup-tests
- /1.0/instances/prime-wombat?project=pg_basebackup-tests
- /1.0/instances/proud-raptor?project=pg_basebackup-tests
- /1.0/instances/rapid-mule?project=pg_basebackup-tests
- /1.0/instances/ready-mite?project=pg_basebackup-tests
- /1.0/instances/smooth-quagga?project=pg_basebackup-tests
- /1.0/profiles/default?project=pg_basebackup-tests

Here you can see that the non-default project contains only a default profile for itself, and the 15 VMs added there. The other entities aren’t needed here, or can be accessed in the default project if required.

How do I use LXD projects with MAAS?

You have several options when it comes to using LXD projects with MAAS:

How do I create a new project for MAAS when instantiating a VM host?

How do I create a new VM in the LXD project associated with a VM host – and what happens?

How do I move an existing VM into the LXD project associated with a VM host – and what happens?

We’ve seen what happens if we compose a VM in a VM host – it’s automatically commissioned. But what if we move an existing VM into a LXD project that’s associated with a MAAS VM host? Let’s try it and see.

First, let’s check on an existing VM in one of our other projects:

$ lxc project switch not-maas
$ lxc list
+-----------------+---------+------+------+-----------------+-----------+
|      NAME       |  STATE  | IPV4 | IPV6 |      TYPE       | SNAPSHOTS |
+-----------------+---------+------+------+-----------------+-----------+
| trusty-drake    | STOPPED |      |      | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+------+-----------------+-----------+
| upward-stallion | STOPPED |      |      | CONTAINER       | 0         |
+-----------------+---------+------+------+-----------------+-----------+
$ lxc move trusty-drake trusty-drake --project not-maas --target-project keystone
$ lxc project switch keystone
$ lxc list
+--------------+---------+------+------+-----------------+-----------+
|     NAME     |  STATE  | IPV4 | IPV6 |      TYPE       | SNAPSHOTS |
+--------------+---------+------+------+-----------------+-----------+
| handy-sloth  | STOPPED |      |      | VIRTUAL-MACHINE | 0         |
+--------------+---------+------+------+-----------------+-----------+
| trusty-drake | STOPPED |      |      | VIRTUAL-MACHINE | 0         |
+--------------+---------+------+------+-----------------+-----------+

We can check the status with MAAS, but we’ll find that the machine isn’t recognized. If we turn it on, it will be enlisted by MAAS. Since MAAS doesn’t know about it yet, we need to turn it on with the following command:

lxc start trusty-drake

Nothing happens for a while, but eventually MAAS will discover the machine and attempt to commission it. In fact, since MAAS doesn’t know what power type to use, it completes all the commissioning scripts except 30-maas-01-bmc-config:

Name Tags Result Date Runtime
20-maas-01-install-lldpd node Passed Mon, 19 Apr. 2021 21:42:22 0:00:00
20-maas-02-dhcp-unconfigured-ifaces node Passed Mon, 19 Apr. 2021 21:42:22 0:00:00
30-maas-01-bmc-config bmc-config, node Skipped Mon, 19 Apr. 2021 21:42:22 0:00:00
50-maas-01-commissioning node Passed Mon, 19 Apr. 2021 21:42:23 0:00:00
maas-capture-lldpd node Passed Mon, 19 Apr. 2021 21:43:17 0:00:53
maas-get-fruid-api-data node Passed Mon, 19 Apr. 2021 21:42:26 0:00:00
maas-kernel-cmdline node Passed Mon, 19 Apr. 2021 21:42:25 0:00:01
maas-list-modaliases node Passed Mon, 19 Apr. 2021 21:42:25 0:00:00
maas-lshw node Passed Mon, 19 Apr. 2021 21:42:26 0:00:02
maas-serial-ports node Passed Mon, 19 Apr. 2021 21:42:24 0:00:00
maas-support-info node Passed Mon, 19 Apr. 2021 21:42:25 0:00:01

This machine will sit in the “New” state until you assign it a power type, and enter the correct power parameters.

What happens to my new MAAS project if I delete the VM host?

At some point, you may want to delete your MAAS VM host. You can do so in the following way:

How hard is it to move LXD entities to another project to hide them from MAAS?

Suppose that want to use MAAS with your default LXD project, and that you have a couple of LXD entities in your default project that you don’t want to use with MAAS:

$ lxc list
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
|      NAME       |  STATE  |         IPV4          |                    IPV6                     |      TYPE       | SNAPSHOTS |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
| trusty-drake    | STOPPED |                       |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
| upward-stallion | RUNNING | 10.196.199.194 (eth0) | fd42:ec:5a53:59d2:216:3eff:febf:7fa7 (eth0) | CONTAINER       | 0         |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
| witty-lizard    | STOPPED |                       |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+

In the above example, you want to use witty-lizard with MAAS, but you want to move the other two entities to a project called not-maas. To accomplish this, you first need to create the not-maas project if it doesn’t exist:

$ lxc project create not-maas
Project not-maas created
$ lxc project list
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |       DESCRIPTION       | USED BY |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| default (current)   | YES    | YES      | YES             | YES      | Default LXD project     | 7       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| maas_vm_host_001    | YES    | YES      | YES             | NO       |                         | 1       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| not-maas            | YES    | YES      | YES             | NO       |                         | 1       |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+
| pg_basebackup-tests | NO     | YES      | NO              | NO       | Project managed by MAAS | 16      |
+---------------------+--------+----------+-----------------+----------+-------------------------+---------+

Having done so, you now want to move trusty-drake and upward-stallion to a new project. Let’s tackle trusty-drake first:

$ lxc move trusty-drake trusty-drake --project default --target-project not-maas --verbose
$ lxc list
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
|      NAME       |  STATE  |         IPV4          |                    IPV6                     |      TYPE       | SNAPSHOTS |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
| upward-stallion | RUNNING | 10.196.199.194 (eth0) | fd42:ec:5a53:59d2:216:3eff:febf:7fa7 (eth0) | CONTAINER       | 0         |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
| witty-lizard    | STOPPED |                       |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
$ lxc project switch not-maas
$ lxc list
+--------------+---------+------+------+-----------------+-----------+
|     NAME     |  STATE  | IPV4 | IPV6 |      TYPE       | SNAPSHOTS |
+--------------+---------+------+------+-----------------+-----------+
| trusty-drake | STOPPED |      |      | VIRTUAL-MACHINE | 0         |
+--------------+---------+------+------+-----------------+-----------+

It’s important to note that the move step may take 30 seconds or more; that’s normal.

Next, let’s try moving upward-stallion, which is a running container:

$ lxc move upward-stallion upward-stallion --project default --target-project not-maas --verbose
Error: Failed creating instance record: Failed initialising instance: Invalid devices: Failed detecting root disk device: No root device could be found
$ lxc list
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
|      NAME       |  STATE  |         IPV4          |                    IPV6                     |      TYPE       | SNAPSHOTS |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
| upward-stallion | RUNNING | 10.196.199.216 (eth0) | fd42:ec:5a53:59d2:216:3eff:fe64:a206 (eth0) | CONTAINER       | 0         |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+
| witty-lizard    | STOPPED |                       |                                             | VIRTUAL-MACHINE | 0         |
+-----------------+---------+-----------------------+---------------------------------------------+-----------------+-----------+

Hmm, what’s that error message about? Well, you actually need to add the default storage pool to the mix, with this command:

lxc profile device add default root disk path=/ pool=default

Having done so, you can try the move again:

$ lxc move upward-stallion upward-stallion --project default --target-project not-maas --verbose
$ lxc list                            
+--------------+---------+------+------+-----------------+-----------+
|     NAME     |  STATE  | IPV4 | IPV6 |      TYPE       | SNAPSHOTS |
+--------------+---------+------+------+-----------------+-----------+
| witty-lizard | STOPPED |      |      | VIRTUAL-MACHINE | 0         |
+--------------+---------+------+------+-----------------+-----------+
$ lxc project switch not-maas
$ lxc list
+-----------------+---------+------+------+-----------------+-----------+
|      NAME       |  STATE  | IPV4 | IPV6 |      TYPE       | SNAPSHOTS |
+-----------------+---------+------+------+-----------------+-----------+
| trusty-drake    | STOPPED |      |      | VIRTUAL-MACHINE | 0         |
+-----------------+---------+------+------+-----------------+-----------+
| upward-stallion | STOPPED |      |      | CONTAINER       | 0         |
+-----------------+---------+------+------+-----------------+-----------+

The move succeeds this time – with an important distinction: the compartment upward-stallion was STOPPED by lxc during the move. This is an important planning consideration when you’re trying to create MAAS VMs & VM hosts in an already-active LXD instantiation. We’ll cover that case in best practices, below.

What are the best practices for using LXD projects with MAAS?

{N} procedures for using LXD projects with MAAS:

Where can I find relevant, detailed, reference material?