No cores and memory accepted when creating VM on LXD host

@dillon-n, can you run a pods read command and let me know the output? I’m specifically intertested in the memory_over_commit_ratio, used, available, and cpu_over_commit_ratio sections. Thank you!

@james-o-benson it doesn’t look like I have the pods command, is that an LXD-specific thing? I’m currently using libvirt due to a different bug MAAS KVM Host Fails Deployment w/error in rackd.log

No it should work with LXD or libvirt. I also can’t deploy LXD due to the timeout. And libvirt hosts deploys but commissioning the VM times out. I had the same error as above, what I noticed is that the GUI showed one thing but CLI showed another. See: 

Ah I see, I was thinking pods was an actual executable command on the VM host, but it’s a subcommand on the MAAS CLI, which I haven’t delved into much yet. Here’s the relevant output you’re looking for:

{
    "memory_over_commit_ratio": 1.3,
    "cpu_over_commit_ratio": 4.0,
    "version": "6.0.0",
    "available": {
        "cores": -1,
        "memory": -2048,
        "local_storage": 236466909184
    },
    "total": {
        "cores": 0,
        "memory": 0,
        "local_storage": 244466909184
    },
    "used": {
        "cores": 1,
        "memory": 2048,
        "local_storage": 8000000000
    },
    "type": "virsh",
}

The ratios there are the same as the ones I set in the UI. However available and total do NOT match, obviously the system has more than 0 cores and memory :slight_smile:.

Really those numbers should be 20 and 160GiB respectively, which is how they display in the UI. I do have one VM deployed with 1C/2GiB, so used is correct. And it looks like available is just substracting used from total.

Storage is correct in all cases.

E: and I just realized I can’t create any more VMs than the one that’s already deployed, presumably because available is negative :frowning:

@billwear This sounds like the same bug I have and filed a bug report on.

1 Like

Try to do a ‘refresh’, you can do that in the GUI under that specific nodes actions in the KVM menu. For me, that fixed some of the resource issues, but also try to do a deploy. Because for me, I was able to create VMs, but none of them ever pass commissioning stage.

EDIT: That fixes the resources some of the time, but not always.

I am having the same issue on MAAS 3.2.6 using a refreshly deployed machine. Due to a bug in 3.2.6, I can’t use maas to initialize LXD on the host so I have to manually do it with LXD init.

VM Host:

$ lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: 
Do you want to configure a new storage pool? (yes/no) [default=yes]: 
Name of the new storage pool [default=default]: 
Name of the storage backend to use (dir, lvm, zfs, btrfs, ceph, cephobject) [default=zfs]: dir
Would you like to connect to a MAAS server? (yes/no) [default=no]: yes
What's the name of this host in MAAS? [default=smoothwall-lxc]: 
URL of your MAAS server (e.g. http://1.2.3.4:5240/MAAS): http://10.160.1.10:5240/MAAS
API key for your MAAS server: <redacted>
Would you like to create a new local network bridge? (yes/no) [default=yes]: 
What should the new bridge be called? [default=lxdbr0]: 
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
Would you like the LXD server to be available over the network? (yes/no) [default=no]: yes
Address to bind LXD to (not including port) [default=all]: 
Port to bind LXD to [default=8443]: 
Trust password for new clients: 
Again: 
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]: 
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: 

MAAS:

$ maas ben vm-hosts read | jq '.[] | select (.name=="smoothwall-lxc")'
{
  "available": {
    "cores": 0,
    "memory": 0,
    "local_storage": 471146233856
  },
  "capabilities": [
    "composable",
    "dynamic_local_storage",
    "over_commit",
    "storage_pools"
  ],
  "total": {
    "cores": 0,
    "memory": 0,
    "local_storage": 471146233856
  },
  "zone": {
    "name": "default",
    "description": "",
    "id": 1,
    "resource_uri": "/MAAS/api/2.0/zones/default/"
  },
  "architectures": [
    "amd64/generic"
  ],
  "cpu_over_commit_ratio": 1,
  "memory_over_commit_ratio": 1,
  "storage_pools": [
    {
      "id": "default",
      "name": "default",
      "type": "dir",
      "path": "/var/snap/lxd/common/lxd/storage-pools/default",
      "total": 471146233856,
      "used": 0,
      "available": 471146233856,
      "default": true
    }
  ],
  "host": {
    "system_id": "qnmbfd",
    "__incomplete__": true
  },
  "tags": [
    "pod-console-logging"
  ],
  "version": "5.6",
  "pool": {
    "name": "default",
    "description": "Default pool",
    "id": 0,
    "resource_uri": "/MAAS/api/2.0/resourcepool/0/"
  },
  "used": {
    "cores": 0,
    "memory": 0,
    "local_storage": 0
  },
  "type": "lxd",
  "name": "smoothwall-lxc",
  "default_macvlan_mode": null,
  "id": 9,
  "resource_uri": "/MAAS/api/2.0/vm-hosts/9/"
}
$ maas ben vm-host compose 9 hostname=juju-controller cores=2 memory=4096 storage='bootvol:40(default)' interfaces='eth0:space=admin-space;eth1:space=data-space'
{"cores": ["Ensure this value is less than or equal to 0."], "memory": ["Ensure this value is less than or equal to 0."]}

I thought this answer should be no because of the Q&A on their website:

Would you like to configure LXD to use an existing bridge or host interface? (yes/no) [default=no]: yes - The bridge LXD creates is isolated and not managed by MAAS. If this bridge is used, you would be able to add the LXD VM host and compose virtual machines, but commissioning, deploying, and any other MAAS action which uses the network will fail – so yes is the correct answer here.

Interesting point to say the least and I’ll certainly keep that in mind in the future, but it doesn’t seem relevant to the topic. Perhaps I’m misunderstanding you in how this could effect maas from recognizing cpu and memory resource. For sanity’s sake, I went ahead and redeployed the same machine without creating a new bridge and have LXD use the maas created bridge and i still ran into the same issue where maas wouldn’t let me create the VM. Would you mind explaining what your thought process is on the LXD netwoking setup? Thanks!

wait, the doc already recommends doing lxd init from the command line, not MAAS. how are you initializing LXD from MAAS?

note that there are also some DHCP settings in that section that you need to confirm or change.

@bjornrun, can you show me the whole screen where you’re getting this error?

I did use LXD from the command line to initialize it on the host.

  1. Deploy bare-metal machine using Ubuntu 20.04 via MAAS Web UI (not registering as LXD host)
  2. ssh into the machine and switch to root user
  3. lxd init
  4. Follow the prompts.
  5. Add the lxd host into maas from the maas kvm tab.

Basically, I was letting lxd create a network bridge when the doc you referenced recommended using an existing bridge that has access to the maas network. (in my case br-ex1)

However, I still don’t see how this can be relevant to this particular issue since maas doesn’t let me create the vm on the host.

Hence:

$ maas ben vm-host compose 9 hostname=juju-controller cores=2 memory=4096 storage='bootvol:40(default)' interfaces='eth0:space=admin-space;eth1:space=data-space'
{"cores": ["Ensure this value is less than or equal to 0."], "memory": ["Ensure this value is less than or equal to 0."]}

If it was a networking issue, I would expect maas to let me create the vm but it would fail during commissioning.

Sorry for the confusion and hopefully that clears it up :+1:

1 Like

@ben-ihelputech, i’ve seen this before. i’ll try to look into this during this week.

1 Like

Hi @ben-ihelputech, I was able to deploy a LXD host manually and connect it to MaaS without issue.

My ‘rough’ process was this:

  1. In MaaS, deploy Ubuntu 20.04 on a physical server utilizing bridge interfaces to the networks you wish your VM’s to connect to.
  2. SSH into the server, issue:
sudo apt update && sudo apt upgrade
sudo apt-get purge -y *lxd* *lxc*
sudo apt-get autoremove -y
sudo snap install lxd
sudo snap refresh lxd --channel=5.6/stable
# When I issued install lxd, it installed version 4.0.9; branch - 4.0/stable, so I had to do a refresh to get the latest.
sudo lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: yes
What IP address or DNS name should be used to reach this node? [default=<REDACTED>]:
Are you joining an existing cluster? (yes/no) [default=no]:
What name should be used to identify this node in the cluster? [default=<SERVER NAME>]:
Setup password authentication on the cluster? (yes/no) [default=no]:
Do you want to configure a new local storage pool? (yes/no) [default=yes]:
Name of the storage backend to use (dir, lvm, zfs, btrfs) [default=zfs]: dir
Do you want to configure a new remote storage pool? (yes/no) [default=no]:
Would you like to connect to a MAAS server? (yes/no) [default=no]: yes
What's the name of this host in MAAS? [default=<SERVER NAME>]:
URL of your MAAS server (e.g. http://1.2.3.4:5240/MAAS): http://<MASS IP>:5240/MAAS
API key for your MAAS server: <REDACTED>
Would you like to configure LXD to use an existing bridge or host interface? (yes/no) [default=no]: yes
Name of the existing bridge or host interface: br0
Is this interface connected to your MAAS server? (yes/no) [default=yes]:
MAAS IPv4 subnet name for this interface (empty for no subnet): <SUBNET>/24
MAAS IPv6 subnet name for this interface (empty for no subnet):
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:
  1. Once finished, I logged into MaaS, KVM tab, ‘Add LXD Host’. This gave me a LXD key to trust and respective command to issue (copy/paste from gui to server): lxc config trust add - <<EOF ....

Once issued on the physical server, the host was successfully added to MaaS and so far, I was able to virtualize resources on MaaS, commission & deploy VMs without issues so far.

@ben-ihelputech, when you say, “1. Deploy machine using machine on ubuntu focal with maas” – wdym?

I just used the MAAS web ui to deploy 3 bare metal host using ubuntu 20.04 as the operating system. Looks like I had a typo in the previous post now that I have re-read it. Sorry for the confusion.

1 Like

okay, so @ben-ihelputech, let’s reconnect on this issue: solved? ongoing?

Haven’t tested on the most recent latest/stable of MaaS or LXD. Went a different route at the time.

1 Like

@billwear I retested this issue and was not able to reproduce it in the latest stable versions of maas and LXD. I’m going to assume it is safe to mark it as resolved on my end.

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.