Spaces vs. fabrics, and what they do

Development
, , , ,
1

Spaces and fabrics are nothing more than network groupings. They are used to group network elements together:

  1. Fabrics group VLANs, so they act like a Layer 2 (Link Layer) category.
  2. Spaces group subnets, so they act like a Layer 3 (Network Layer) category.
  3. In fact, the mechanics are much more complicated, so (1) and (2) are useful generalisations only.

Let’s get about proving that to ourselves.

Spaces are created without association

In MAAS, spaces are created without any association with any other network element, as this help listing shows:

usage: maas admin spaces create [--help] [-d] [-k] [data [data ...]]

Create a space


This method accepts keyword arguments.  Pass each argument as a
key-value pair with an equals sign between the key and the value:
key1=value1 key2=value key3=value3.  Keyword arguments must come after
any positional arguments.

Create a new space.

:param name: Required.  The name of the new space.
:type name: String

 :param description: Optional.  A description of the new
space.
:type description: String


Common command-line options:
    --help, -h
	Show this help message and exit.
    -d, --debug
	Display more information about API responses.
    -k, --insecure
	Disable SSL certificate check

You can infer that spaces do not carry associations themselves, due to the lack of any named parameters that would let you add those associations.

Subnets can join a space; they are always in a VLAN and a fabric

You can associate a space with a subnet like this:

usage: maas admin subnet update [--help] [-d] [-k] id [data [data ...]]

Update a subnet


Positional arguments:
	id


This method accepts keyword arguments.  Pass each argument as a
key-value pair with an equals sign between the key and the value:
key1=value1 key2=value key3=value3.  Keyword arguments must come after
any positional arguments.

Update a subnet with the given ID.

:param cidr: Optional.  The network CIDR for this
subnet.
:type cidr: String

 :param name: Optional.  The subnet's name.
:type name: String

 :param description: Optional.  The subnet's
description.
:type description: String

 :param vlan: Optional.  VLAN this subnet belongs to.
Defaults to the default VLAN for the provided fabric or defaults to the
default VLAN in the default fabric (if unspecified).
:type vlan: String

 :param fabric: Optional.  Fabric for the subnet.
Defaults to the fabric the provided VLAN belongs to, or defaults to the
default fabric.
:type fabric: String

 :param vid: Optional.  VID of the VLAN this subnet belongs
to. Only used when vlan is not provided. Picks the VLAN with this VID
in the provided fabric or the default fabric if one is not given.
:type vid: Int

 :param space: Optional.  Space this subnet is in.
Defaults to the default space.
:type space: String

 :param gateway_ip: Optional.  The gateway IP address
for this subnet.
:type gateway_ip: String

 :param rdns_mode: Optional.  How reverse
DNS is handled for this subnet.  One of:

- ``0`` Disabled: No reverse zone is created.
- ``1`` Enabled: Generate reverse zone.
- ``2`` RFC2317: Extends '1' to create the necessary parent zone with
  the appropriate CNAME resource records for the network, if the the
  network is small enough to require the support described in RFC2317.
:type rdns_mode: Int

 :param allow_dns: Optional.  Configure MAAS DNS to allow
DNS resolution from this subnet. '0' == False,'1' == True.
:type allow_dns: Int

 :param allow_proxy: Optional.  Configure maas-proxy to
allow requests from this subnet. '0' == False, '1' == True.
:type allow_proxy: Int

 :param dns_servers: Optional.  Comma-separated list of
DNS servers for this subnet.
:type dns_servers: String

 :param managed: Optional.  In MAAS 2.0+,
all subnets are assumed to be managed by default.
:type managed: Int

 :param disabled_boot_architectures: Optional.  A comma
or space separated list of boot architectures which will not be
responded to by isc-dhcpd. Values may be the MAAS name for the boot
architecture, the IANA hex value, or the isc-dhcpd octet.

Only managed subnets allow DHCP to be enabled on their related dynamic
ranges. (Thus, dynamic ranges become "informational only"; an
indication that another DHCP server is currently handling them, or that
MAAS will handle them when the subnet is enabled for management.)

Managed subnets do not allow IP allocation by default. The meaning of a
"reserved" IP range is reversed for an unmanaged subnet. (That is, for
managed subnets, "reserved" means "MAAS cannot allocate any IP address
within this reserved block". For unmanaged subnets, "reserved" means
"MAAS must allocate IP addresses only from reserved IP ranges."
:type disabled_boot_architectures: String


Common command-line options:
    --help, -h
	Show this help message and exit.
    -d, --debug
	Display more information about API responses.
    -k, --insecure
	Disable SSL certificate check

Note the optional parameter space in the list:

 :param space: Optional.  Space this subnet is in.
Defaults to the default space.
:type space: String

This is a little bit misleading, because if you haven’t defined a space, the default space is undefined, which is, in reality, not a space.

You might also note the optional parameter fabric in this list:

 :param fabric: Optional.  Fabric for the subnet.
Defaults to the fabric the provided VLAN belongs to, or defaults to the
default fabric.
:type fabric: String

In this case, the fabric does default to the default fabric, which is usually fabric-0.

In MAAS, VLANs always exist in some fabric, but can join a space

You can also associate a VLAN with a space, like this:

usage: maas admin vlan update [--help] [-d] [-k]
                              fabric_id vid [data [data ...]]

Update VLAN


Positional arguments:
	fabric_id
	vid


This method accepts keyword arguments.  Pass each argument as a
key-value pair with an equals sign between the key and the value:
key1=value1 key2=value key3=value3.  Keyword arguments must come after
any positional arguments.

Updates a given VLAN.

:param name: Optional.  Name of the VLAN.
:type name: String

 :param description: Optional.  Description of the VLAN.
:type description: String

 :param mtu: Optional.  The MTU to use on the VLAN.
:type mtu: Int

 :param dhcp_on: Optional.  Whether or not DHCP should
be managed on the VLAN.
:type dhcp_on: Boolean

 :param primary_rack: Optional.  The primary rack
controller managing the VLAN (system_id).
:type primary_rack: String

 :param secondary_rack: Optional.  The secondary rack
controller managing the VLAN (system_id).
:type secondary_rack: String

 :param relay_vlan: Optional.  Relay VLAN ID. Only set when
this VLAN will be using a DHCP relay to forward DHCP requests to
another VLAN that MAAS is managing. MAAS will not run the DHCP relay
itself, it must be configured to proxy requests to the primary and/or
secondary rack controller interfaces for the VLAN specified in this
field.
:type relay_vlan: Int

 :param space: Optional.  The space this VLAN should be
placed in. Passing in an empty string (or the string 'undefined') will
cause the VLAN to be placed in the 'undefined' space.
:type space: String


Common command-line options:
    --help, -h
	Show this help message and exit.
    -d, --debug
	Display more information about API responses.
    -k, --insecure
	Disable SSL certificate check

Again, the operative parameter is here:

:param space: Optional.  The space this VLAN should be
placed in. Passing in an empty string (or the string 'undefined') will
cause the VLAN to be placed in the 'undefined' space.
:type space: String

But note that the fabric_id is required to update the VLAN. It is also required to create the VLAN:

usage: maas admin vlans create [--help] [-d] [-k] fabric_id [data [data ...]]

Create a VLAN


Positional arguments:
	fabric_id


This method accepts keyword arguments.  Pass each argument as a
key-value pair with an equals sign between the key and the value:
key1=value1 key2=value key3=value3.  Keyword arguments must come after
any positional arguments.

Creates a new VLAN.

:param name: Optional.  Name of the VLAN.
:type name: String

 :param description: Optional.  Description of the new
VLAN.
:type description: String

 :param vid: Required.  VLAN ID of the new VLAN.
:type vid: Int

 :param mtu: Optional.  The MTU to use on the VLAN.
:type mtu: Int

 :param space: Optional.  The space this VLAN should be
placed in. Passing in an empty string (or the string 'undefined') will
cause the VLAN to be placed in the 'undefined' space.
:type space: String


Common command-line options:
    --help, -h
	Show this help message and exit.
    -d, --debug
	Display more information about API responses.
    -k, --insecure
	Disable SSL certificate check

In other words, within MAAS, VLANs always exist within a fabric, even if it’s the default fabric.

No other MAAS element can be assigned to a space or fabric

Using the CLI help and parameters as a guide, nothing else in the maas admin -h listing can be associated with a space or a fabric, other than what’s listed above.

A fabric can group VLANs

We can test this with the following commands.

Create a unique fabric

stormrider@cloudburst:~$ maas admin fabrics create name="new-fabric"
Success.
Machine-readable output follows:
{
    "class_type": null,
    "id": 62,
    "name": "new-fabric",
    "vlans": [
        {
            "vid": 0,
            "mtu": 1500,
            "dhcp_on": false,
            "external_dhcp": null,
            "relay_vlan": null,
            "secondary_rack": null,
            "id": 5064,
            "space": "undefined",
            "primary_rack": null,
            "fabric": "new-fabric",
            "fabric_id": 62,
            "name": "untagged",
            "resource_uri": "/MAAS/api/2.0/vlans/5064/"
        }
    ],
    "resource_uri": "/MAAS/api/2.0/fabrics/62/"
}

Note that this fabric is automatically assigned to the default VLAN (vid 0, aka untagged). So in reality, a fabric can’t exist in MAAS separate from some VLAN.

Create a new, unique subnet

stormrider@cloudburst:~$ maas admin subnets create cidr="10.194.15.0/24"
Success.
Machine-readable output follows:
{
    "name": "10.194.15.0/24",
    "description": "",
    "vlan": {
        "vid": 0,
        "mtu": 1500,
        "dhcp_on": false,
        "external_dhcp": null,
        "relay_vlan": null,
        "secondary_rack": null,
        "id": 5004,
        "space": "undefined",
        "primary_rack": null,
        "fabric": "fabric-0",
        "fabric_id": 0,
        "name": "untagged",
        "resource_uri": "/MAAS/api/2.0/vlans/5004/"
    },
    "cidr": "10.194.15.0/24",
    "rdns_mode": 2,
    "gateway_ip": "",
    "dns_servers": [],
    "allow_dns": true,
    "allow_proxy": true,
    "active_discovery": false,
    "managed": true,
    "disabled_boot_architectures": [],
    "id": 12,
    "space": "undefined",
    "resource_uri": "/MAAS/api/2.0/subnets/12/"
}

Note that this subnet is also assigned to the default VLAN when created. This means that every fabric always has an associated VLAN, and every subnet always has an associated VLAN. But look at what we just did. We have a subnet on the untagged VLAN associated with fabric-0, and we also have the untagged VLAN associated with fabric-62, aka new-fabric. So two different fabrics can access the same VID (but reach different subnets by so doing).

Assign the new fabric and subnet separately

The fabric is already associated with the VLAN, so we just need to do this:

stormrider@cloudburst:~$ maas admin subnet update 12 fabric=60
Success.
Machine-readable output follows:
{
    "name": "10.194.15.0/24",
    "description": "",
    "vlan": {
        "vid": 0,
        "mtu": 1500,
        "dhcp_on": false,
        "external_dhcp": null,
        "relay_vlan": null,
        "secondary_rack": null,
        "id": 5004,
        "space": "undefined",
        "primary_rack": null,
        "fabric": "fabric-0",
        "fabric_id": 0,
        "name": "untagged",
        "resource_uri": "/MAAS/api/2.0/vlans/5004/"
    },
    "cidr": "10.194.15.0/24",
    "rdns_mode": 2,
    "gateway_ip": "",
    "dns_servers": [],
    "allow_dns": true,
    "allow_proxy": true,
    "active_discovery": false,
    "managed": true,
    "disabled_boot_architectures": [],
    "id": 12,
    "space": "undefined",
    "resource_uri": "/MAAS/api/2.0/subnets/12/"
}

This returns information as if it worked, with a “success” message, but in fact did not change the fabric of this subnet. You’ll see that it remains in fabric-0. We can understand how this works by trying this in the UI. If we use the UI to assign a subnet to a fabric, we are required to choose a specific VLAN for the subnet in order for the update to complete. Otherwise, MAAS just ignores the change, returns no error message, does not change the subnet associated with a fabric, and goes on about its business.

What we just learned

So here’s the logic associated with what we just did:

  1. A subnet always has a VLAN, even if it’s the default.
  2. A VLAN always has a fabric, even if it’s the default.
  3. Therefore, a subnet is always reachable thru a fabric.

You can confirm this by listing the contents of, say, fabric=60, like this:

Success.
Machine-readable output follows:
{
    "class_type": null,
    "vlans": [
        {
            "vid": 0,
            "mtu": 1500,
            "dhcp_on": false,
            "external_dhcp": null,
            "relay_vlan": null,
            "secondary_rack": null,
            "name": "untagged",
            "fabric_id": 60,
            "space": "undefined",
            "fabric": "fabric-one",
            "primary_rack": null,
            "id": 5061,
            "resource_uri": "/MAAS/api/2.0/vlans/5061/"
        },
        {
            "vid": 100,
            "mtu": 1500,
            "dhcp_on": false,
            "external_dhcp": null,
            "relay_vlan": null,
            "secondary_rack": null,
            "name": "vlan-alpha",
            "fabric_id": 60,
            "space": "undefined",
            "fabric": "fabric-one",
            "primary_rack": null,
            "id": 5062,
            "resource_uri": "/MAAS/api/2.0/vlans/5062/"
        }
    ],
    "name": "fabric-one",
    "id": 60,
    "resource_uri": "/MAAS/api/2.0/fabrics/60/"
}

Note that this doesn’t actually show any subnets, only VLANs. So fabrics group VLANS, not subnets, and thus they would generally be considered a Layer 2 construct, since by inference only, they depend on the P/Q tags in the Ethernet frame (the layer two PDU).

VLANs without subnets?

Okay, can we create a VLAN that has no subnets? Well, let’s see. The help for VLAN creation tells us this:

stormrider@cloudburst:~$ maas admin vlans create -h
usage: maas admin vlans create [--help] [-d] [-k] fabric_id [data [data ...]]

Create a VLAN


Positional arguments:
	fabric_id


This method accepts keyword arguments.  Pass each argument as a
key-value pair with an equals sign between the key and the value:
key1=value1 key2=value key3=value3.  Keyword arguments must come after
any positional arguments.

Creates a new VLAN.

:param name: Optional.  Name of the VLAN.
:type name: String

 :param description: Optional.  Description of the new
VLAN.
:type description: String

 :param vid: Required.  VLAN ID of the new VLAN.
:type vid: Int

 :param mtu: Optional.  The MTU to use on the VLAN.
:type mtu: Int

 :param space: Optional.  The space this VLAN should be
placed in. Passing in an empty string (or the string 'undefined') will
cause the VLAN to be placed in the 'undefined' space.
:type space: String


Common command-line options:
    --help, -h
	Show this help message and exit.
    -d, --debug
	Display more information about API responses.
    -k, --insecure
	Disable SSL certificate check

Creating an empty VLAN?

So if we issue a command like this:

stormrider@cloudburst:~$ maas admin vlans create 61 vid=4000
Success.
Machine-readable output follows:
{
    "vid": 4000,
    "mtu": 1500,
    "dhcp_on": false,
    "external_dhcp": null,
    "relay_vlan": null,
    "id": 5065,
    "name": "4000",
    "fabric": "fabric-two",
    "primary_rack": null,
    "secondary_rack": null,
    "fabric_id": 61,
    "space": "undefined",
    "resource_uri": "/MAAS/api/2.0/vlans/5065/"
}

…it appears to work. If we take a look at the MAAS UI, we’ll see that fabric-two (id=61) has a VLAN with vid=4000, but no associated subnet. So currently, fabric-two can reach two VLANs, but no subnets.

Grouping things with spaces

Okay, so let’s try these same exercises with spaces. We already created a space called space-1, which we have available to connect with subnets and VLANs. Let’s try something – let’s associate a specific subnet on the default VLAN with a space, but not directly associate it with the VLAN, and see what happens.

The command looks like this:

stormrider@cloudburst:~$ maas admin subnet update 13 space="space-1"
{"space": ["Spaces may no longer be set on subnets. Set the space on the underlying VLAN."]}

Oops. So this used to work, but it doesn’t any more. Well, the only other option is a VLAN, so let’s try that command:

stormrider@cloudburst:~$ maas admin vlan update 61 0 space=1
Success.
Machine-readable output follows:
{
    "vid": 0,
    "mtu": 1500,
    "dhcp_on": false,
    "external_dhcp": null,
    "relay_vlan": null,
    "secondary_rack": null,
    "id": 5063,
    "space": "space1",
    "primary_rack": null,
    "fabric": "fabric-two",
    "fabric_id": 61,
    "name": "untagged",
    "resource_uri": "/MAAS/api/2.0/vlans/5063/"
}

So, note that we had to put in both a fabric id and a VLAN id in order to assign a space. Both coordinates are required to correctly locate the entity to be included in the space. Also notice that there doesn’t have to be a subnet associated with this VLAN – although without a subnet, the space can’t actually do anything useful when it’s addressed by Juju.

Okay, this command appears to work, and we can confirm it with the UI. If we look at the untagged VLAN (vid=0) in any other fabric (like, for example, in fabric-0), the space is still undefined. But if we look at the untagged VLAN (same vid) in fabric-two (id=61), we see the space is assigned to space1. Even VLAN 4000, which is also in fabric-two, doesn’t have a defined space.

So if we add a subnet to the untagged VLAN in fabric-two, we’ll be able to reach only that subnet with space1, even though it’s still the untagged VLAN. Before we do that, let’s do a little detective work on space1:

stormrider@cloudburst:~$ maas admin space read 1
Success.
Machine-readable output follows:
{
    "resource_uri": "/MAAS/api/2.0/spaces/1/",
    "id": 1,
    "name": "space1",
    "vlans": [
        {
            "vid": 0,
            "mtu": 1500,
            "dhcp_on": false,
            "external_dhcp": null,
            "relay_vlan": null,
            "id": 5063,
            "name": "untagged",
            "fabric": "fabric-two",
            "primary_rack": null,
            "secondary_rack": null,
            "fabric_id": 61,
            "space": "space1",
            "resource_uri": "/MAAS/api/2.0/vlans/5063/"
        }
    ],
    "subnets": []
}

Yep, that’s what’s happening. space1 addresses only those subnets that are part of the untagged VLAN and also in fabric-two. So a fabric matters for a space, but a space means nothing to a fabric.

Okay, let’s test that subnet thing: let’s create a subnet on fabric-two:

stormrider@cloudburst:~$ maas admin subnets create cidr="10.198.16.0/24" fabric=61
Success.
Machine-readable output follows:
{
    "name": "10.198.16.0/24",
    "description": "",
    "vlan": {
        "vid": 0,
        "mtu": 1500,
        "dhcp_on": false,
        "external_dhcp": null,
        "relay_vlan": null,
        "id": 5063,
        "name": "untagged",
        "fabric": "fabric-two",
        "primary_rack": null,
        "secondary_rack": null,
        "fabric_id": 61,
        "space": "space1",
        "resource_uri": "/MAAS/api/2.0/vlans/5063/"
    },
    "cidr": "10.198.16.0/24",
    "rdns_mode": 2,
    "gateway_ip": "",
    "dns_servers": [],
    "allow_dns": true,
    "allow_proxy": true,
    "active_discovery": false,
    "managed": true,
    "disabled_boot_architectures": [],
    "id": 15,
    "space": "space1",
    "resource_uri": "/MAAS/api/2.0/subnets/15/"
}

And let’s check the UI to see if what we thought would happen actually happened. Yep, that’s what happened. If I look at space1 in the UI, it shows me a list of subnets in this space. I can do the same thing by repeating the above command:

stormrider@cloudburst:~$ maas admin space read 1
Success.
Machine-readable output follows:
{
    "vlans": [
        {
            "vid": 0,
            "mtu": 1500,
            "dhcp_on": false,
            "external_dhcp": null,
            "relay_vlan": null,
            "secondary_rack": null,
            "id": 5063,
            "space": "space1",
            "primary_rack": null,
            "fabric": "fabric-two",
            "fabric_id": 61,
            "name": "untagged",
            "resource_uri": "/MAAS/api/2.0/vlans/5063/"
        }
    ],
    "id": 1,
    "subnets": [
        {
            "name": "10.198.16.0/24",
            "description": "",
            "vlan": {
                "vid": 0,
                "mtu": 1500,
                "dhcp_on": false,
                "external_dhcp": null,
                "relay_vlan": null,
                "secondary_rack": null,
                "id": 5063,
                "space": "space1",
                "primary_rack": null,
                "fabric": "fabric-two",
                "fabric_id": 61,
                "name": "untagged",
                "resource_uri": "/MAAS/api/2.0/vlans/5063/"
            },
            "cidr": "10.198.16.0/24",
            "rdns_mode": 2,
            "gateway_ip": null,
            "dns_servers": [],
            "allow_dns": true,
            "allow_proxy": true,
            "active_discovery": false,
            "managed": true,
            "disabled_boot_architectures": [],
            "id": 15,
            "space": "space1",
            "resource_uri": "/MAAS/api/2.0/subnets/15/"
        }
    ],
    "name": "space1",
    "resource_uri": "/MAAS/api/2.0/spaces/1/"
}

So, in fact, the Juju statement that spaces group subnets is true. And it’s also true that fabrics essentially group VLANs – except that fabrics are required to pinpoint spaces.

What can we conclude?

So if you study this carefully, you’ll see that the following global statements are true:

  1. VLANs are always in a fabric, by default.
  2. Subnets are always in a VLAN, by default.
  3. Fabrics group VLANs, so they act like a Layer 2 (Link Layer) category.
  4. Spaces group subnets, so they act like a Layer 3 (Network Layer) category. But the mechanics of this are, in fact, slightly more complicated, because spaces select subnets based on the VLAN and fabric.

How to use this information

So what can we do with this? Well:

  1. If you want to create a group of specific VLANs, create a new fabric and put them all in that fabric.

  2. If you want to create a group of specific subnets, make sure they can be independently selected by fabric/VLAN combinations; then create a new space and associate that space with the relevant combinations.

There you have it: spaces vs. fabrics.

7 Likes
2

imo version of this should go into the networking docs :slight_smile:

2 Likes
3

yup. (gratuitous words to make it 20 characters, which ruins the mic drop). :slight_smile:

4

Note that this doesn’t actually show any subnets, only VLANs. So fabrics group VLANS, not subnets, and thus they would generally be considered a Layer 2 construct, since by inference only, they depend on the P/Q tags in the Ethernet frame (the layer two PDU).

To further clarify, Fabrics are an L1 physical link layer concept, Vlans are L2 data-link layer, Subnets are L3 routable abstractions. Spaces in MAAS are a fuzzy L2-ish concept, linked at the L3 subnet layer, but are typically useful for using MAAS as a cloud provider for Juju. Juju requires that the machine attached to the subnet which is tagged with a given space has an IP assigned on the space to be considered to be part of the space. Juju’s concept of spaces is strictly L3, but does allow for multiple L3’s to be logically grouped together as a “space” for access of related or upstream APIs/UIs. The idea of spaces is to tell Juju charm developers which of many network links on a host to use for specific communication wiith peers/consumers.

2 Likes
5

you are welcome to edit this in. no pride of authorship here.

1 Like
6

The way I used it (which may or may not be correct at all) was I associated fabrics with physical locations/data centers then the VLANS and subnets for those said locations. We used space names that corresponded with the physical location and vlan names in our DCIM system. I.e. _core_services or similar.

1 Like
7

did it do what you wanted? if so, kudos.

8

It worked quite well. We had a couple of wrenches thrown because spaces associate to VLANS not subnets but that was not a major problem so far.

1 Like
9

right, they group subnets, but associate to VLANs. that’s the confusing part. kinda like ARP when you first encounter it: which layer does it associate with?