KVM Pod Networking with macvlan


#1

KVM Pod Networking

The purpose of this document is to give users some insight into some of the steps carried out when testing KVM Pod networking functionality that was introduced in the 18.10 product cycle for MAAS.

Setting up MAAS in LXD Container with Libvirt

For testing the KVM Pod networking functionality to its full extent we will need to setup a LXD container where we will host MAAS and libvirt. For more details on how to do this, please see Setting up a Flexible "Virtual MAAS" Test Environment

Creating KVM Pod

There are several ways you can create a KVM Pod to get up and started. Some of this is documented in other places within the MAAS documentation but is either briefly contained here or linked to for convenience.

Creating KVM Pod with UI

To create a KVM Pod with the UI

Creating KVM Pod with API

To create a KVM Pod with the API/CLI

Creating KVM Pod with python-libmaas

To create a KVM Pod with python-libmaas follow these steps

Here is an example:

$ bin/maas shell
Welcome to the MAAS shell.

In [1]: from maas.client import login

In [2]: client = login("http://10.0.0.2:5240/MAAS/", username="admin", password="test")

In [3]: pod = client.pods.create(type="virsh", power_address="qemu+ssh://ubuntu@10.0.0.2/system", power_pass="ubuntu")

Updating KVM Pod Host and Default MACVLAN Mode

There are three new additions to MAAS for KVM Pod networking. The first two are Pod specific; the Pod’s host and default MACVLAN mode. These two optional settings can be configured in the UI, API/CLI, and with python-libmaas. Let’s take a look at the different ways to update them and their associated expected outputs.

Updating Pod host and default_macvlan_mode with UI

TODO (this portion has not been implemented yet).

Updating Pod host and default_macvlan_mode with API

Updating the Pod’s host and default_macvlan_mode settings with valid input will succeed:

# maas admin pod update 11 host="gdehta" default_macvlan_mode="private"
Success.
Machine-readable output follows:
{
    "total": {
        "cores": 8,
        "memory": 15918,
        "local_storage": 88341741568
    },
    "host": {
        "system_id": "gdehta",
        "__incomplete__": true
    },
    "id": 11,
    "pool": {
        "name": "default",
        "description": "Default pool",
        "id": 0,
        "resource_uri": "/MAAS/api/2.0/resourcepool/0/"
    },
    "default_macvlan_mode": "private",
    "zone": {
        "name": "default",
        "description": "",
        "id": 1,
        "resource_uri": "/MAAS/api/2.0/zones/default/"
    },
    "architectures": [
        "amd64/generic"
    ],
    "name": "holy-jay",
    "type": "virsh",
    "used": {
        "cores": 3,
        "memory": 3072,
        "local_storage": 24000000000
    },
    "available": {
        "cores": 5,
        "memory": 12846,
        "local_storage": 64341741568
    },
    "storage_pools": [
        {
            "id": "77bf9255-e2f6-4f94-80ef-459cabd966e5",
            "name": "default",
            "type": "dir",
            "path": "/var/lib/libvirt/images",
            "total": 88341741568,
            "used": 24000000000,
            "available": 64341741568,
            "default": true
        }
    ],
    "memory_over_commit_ratio": 1.0,
    "tags": [
        "virtual"
    ],
    "cpu_over_commit_ratio": 1.0,
    "capabilities": [
        "composable",
        "dynamic_local_storage",
        "over_commit",
        "storage_pools"
    ],
    "resource_uri": "/MAAS/api/2.0/pods/11/"
}

As we can see the Pod’s host and default_macvlan_mode were both updated correctly.

Additionally, we can set these Pod settings to empty strings:

# maas admin pod update 11 host="" default_macvlan_mode=""
Success.
Machine-readable output follows:
{
    "cpu_over_commit_ratio": 1.0,
    "total": {
        "cores": 8,
        "memory": 15918,
        "local_storage": 88341741568
    },
    "tags": [
        "virtual"
    ],
    "available": {
        "cores": 5,
        "memory": 12846,
        "local_storage": 64341741568
    },
    "memory_over_commit_ratio": 1.0,
    "id": 11,
    "default_macvlan_mode": "",
    "type": "virsh",
    "capabilities": [
        "composable",
        "dynamic_local_storage",
        "over_commit",
        "storage_pools"
    ],
    "zone": {
        "name": "default",
        "description": "",
        "id": 1,
        "resource_uri": "/MAAS/api/2.0/zones/default/"
    },
    "host": {
        "system_id": null,
        "__incomplete__": true
    },
    "used": {
        "cores": 3,
        "memory": 3072,
        "local_storage": 24000000000
    },
    "architectures": [
        "amd64/generic"
    ],
    "pool": {
        "name": "default",
        "description": "Default pool",
        "id": 0,
        "resource_uri": "/MAAS/api/2.0/resourcepool/0/"
    },
    "storage_pools": [
        {
            "id": "77bf9255-e2f6-4f94-80ef-459cabd966e5",
            "name": "default",
            "type": "dir",
            "path": "/var/lib/libvirt/images",
            "total": 88341741568,
            "used": 24000000000,
            "available": 64341741568,
            "default": true
        }
    ],
    "name": "holy-jay",
    "resource_uri": "/MAAS/api/2.0/pods/11/"
}

It should be noted that the default for default_macvlan_mode is “bridge” when it is set to null or the empty string.

Invalid input for these will result in an error as one would expect:

# maas admin pod update 11 host="yodel" default_macvlan_mode="lady-who"
{"default_macvlan_mode": ["Select a valid choice. lady-who is not one of the available choices."], "host": ["Select a valid choice. yodel is not one of the available choices."]}

Updating Pod host and default_macvlan_mode with python-libmaas

Updating the Pod’s host and default_macvlan_mode settings with valid input will succeed:

TODO - this portion is waiting on this bug to be fixed.
Once this issue is resolved we will be able to update pod.host and show the expected output.

Here is an example of just updating default_macvlan_mode:

$ bin/maas shell
Welcome to the MAAS shell.

In [1]: from maas.client import login

In [2]: client = login("http://10.0.0.2:5240/MAAS/", username="admin", password="test")

In [3]: pod = client.pods.list()[0]

In [4]: pod.default_macvlan_mode
Out[4]: ''

In [5]: pod.default_macvlan_mode = "vepa"

In [6]: pod.save()

In [7]: 

Composing Machines with Interface Constraints

There are three new additions to MAAS for KVM Pod networking. The first two were done in the previous section. The last one is being able to specify interface constraints while composing machines. These interface constraints can be configured in the UI, API/CLI, and with python-libmaas.

Composing Machines with UI

TODO - UI still needs to be implemented to show this

Composing Machines with API

Composing machines with valid interface constraints:

# maas admin pod compose 11 interfaces="eth0:subnet=10.0.0.0/24"
Success.
Machine-readable output follows:
{
    "system_id": "tse88m",
    "resource_uri": "/MAAS/api/2.0/machines/tse88m/"
}

Composing machines with invalid interface constraints:

# maas admin pod compose 11 interfaces="eth0:subnet=19.0.0.0/24"
This pod does not match the specified networks.

Composing Machines with python-libmaas

Composing machines with valid interface constraints:

$ bin/maas shell
Welcome to the MAAS shell.

In [1]: from maas.client import login

In [2]: client = login("http://10.0.0.2:5240/MAAS/", username="admin", password="test")

In [3]: pod = client.pods.list()[0]

In [4]: pod.compose(interfaces="eth1:subnet=10.0.0.0/24")
Out[4]: {'system_id': 'wcfw4q', 'resource_uri': '/MAAS/api/2.0/machines/wcfw4q/'}

In [5]: 

Composing machines with invalid interface constraints:

In [5]: pod.compose(interfaces="eth1:subnet=19.0.0.0/24")
---------------------------------------------------------------------------
CallError                                 Traceback (most recent call last)
<ipython-input-5-782f139f5cc9> in <module>()
----> 1 pod.compose(interfaces="eth1:subnet=19.0.0.0/24")

~/code/canonical/python-libmaas/maas/client/utils/async.py in wrapper(*args, **kwargs)
     47         if not eventloop.is_running():
     48             while isawaitable(result):
---> 49                 result = eventloop.run_until_complete(result)
     50         return result
     51 

/usr/lib/python3.6/asyncio/base_events.py in run_until_complete(self, future)
    466             raise RuntimeError('Event loop stopped before Future completed.')
    467 
--> 468         return future.result()
    469 
    470     def stop(self):

~/code/canonical/python-libmaas/maas/client/viscera/pods.py in compose(self, cores, memory, cpu_speed, architecture, storage, hostname, domain, zone, interfaces)
    231                     "zone must be an int, str or Zone, not %s" %
    232                     type(zone).__name__)
--> 233         return await self._handler.compose(**params, id=self.id)
    234 
    235     async def delete(self):

~/code/canonical/python-libmaas/maas/client/bones/__init__.py in __call__(self, **data)
    300             if key.startswith('_'):
    301                 data[key[1:]] = data.pop(key)
--> 302         response = await self.bind(**params).call(**data)
    303         return response.data
    304 

~/code/canonical/python-libmaas/maas/client/bones/__init__.py in dispatch(self, uri, body, headers)
    461                         "uri": uri,
    462                     }
--> 463                     raise CallError(request, response, content, self)
    464 
    465                 # Decode from JSON if that's what it's declared as.

CallError: POST http://10.0.0.2:5240/MAAS/api/2.0/pods/11/?op=compose -> HTTP 400 Bad Request (This pod does not match the specified networks.)

#2

I think this post doesn’t explain what macvlan vs host networking actually means. We should explain that.