I have a remote machine (that is part of a cluster) where I installed MAAS and LXD.
$ snap list
lxd 5.21.1-2d13beb 28463 latest/stable canonical✓ -
maas 3.4.2-14353-g.5a5221d57 35359 3.4/stable canonical✓ -
It is stated on this page to use an existing bridge instead of using the one that LXD creates (lxdbr0). Thus I decided to create a bridge with netplan as follow:
network:
version: 2
renderer: NetworkManager
ethernets:
eno8303:
dhcp4: no
addresses:
- 10.130.2.77/23
routes:
- to: default
via: 10.130.2.1
nameservers:
addresses: [8.8.8.8]
enp1s0f0np0:
dhcp4: no
match:
macaddress: 04:32:01:40:c0:a0
mtu: 1500
set-name: enp1s0
bridges:
br0:
addresses:
- 10.62.0.10/24
routes:
- to: default
via: 10.62.0.1
table: 2
routing-policy:
- from: 10.62.0.0/24
table: 2
interfaces:
- enp1s0f0np0
macaddress: 04:32:01:40:c0:a0
mtu: 1500
nameservers:
addresses: [8.8.8.8]
search:
- maas
parameters:
forward-delay: 15
stp: false
The physical interface eno8303 is my entry point on that remote machine and connected to internet.
The physical interface enp1s0f0np0 is another nic on which the other machines of the cluster are accessible. Before adding LXD, I used that interface to PXE boot the physical machines with MAAS.
Now, if I am doing it right, I want to use that interface to manage LXD VMs as well, thus I created the bridge br0.
Which works fine. I can successfully add and deploy LXD VMs with MAAS.
However, it seems that the VM’s network is not configured properly. I cannot ping the bridge gateway 10.62.0.1, which is the default route, and thus no internet connectivity in the VMs.
root@mytest:~# ip r
default via 10.62.0.1 dev enp5s0 proto static
10.62.0.0/24 dev enp5s0 proto kernel scope link src 10.62.0.7
root@mytest:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
From 10.62.0.7 icmp_seq=1 Destination Host Unreachable
From 10.62.0.7 icmp_seq=2 Destination Host Unreachable
From 10.62.0.7 icmp_seq=3 Destination Host Unreachable
I am certainly missing something here due to my lack of knowledge with bridges and networking.
$ lxc config show --expanded mytest
architecture: x86_64
config:
limits.cpu: "1"
limits.memory: "2147483648"
limits.memory.hugepages: "false"
security.secureboot: "false"
volatile.cloud-init.instance-id: 664facb6-6353-4bc2-9f6d-92a5d3339f17
volatile.eth0.host_name: tap0c9df063
volatile.eth0.hwaddr: 00:16:3e:26:d3:e3
volatile.last_state.power: RUNNING
volatile.uuid: 6d41b004-37e0-455c-827d-c51d56dacba4
volatile.uuid.generation: 6d41b004-37e0-455c-827d-c51d56dacba4
volatile.vsock_id: "27688414"
devices:
eth0:
boot.priority: "1"
name: eth0
nictype: bridged
parent: br0
type: nic
root:
boot.priority: "0"
path: /
pool: default
size: "8000000000"
type: disk
ephemeral: false
profiles: []
stateful: false
description: ""
$ lxc network show br0
name: br0
description: ""
type: bridge
managed: false
status: ""
config: {}
used_by:
- /1.0/instances/mytest2?project=maas-project
- /1.0/instances/mytest?project=maas-project
- /1.0/profiles/default?project=maas-project
locations: []
$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.130.2.1 0.0.0.0 UG 100 0 0 eno8303
10.62.0.0 0.0.0.0 255.255.255.0 U 425 0 0 br0
10.130.2.0 0.0.0.0 255.255.254.0 U 100 0 0 eno8303
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eno8303
EDIT
Ok I found the solution.
First I used a more simple bridge setup:
bridges:
br0:
addresses:
- 10.62.0.1/24
interfaces:
- enp1s0f0np0
macaddress: 04:32:01:40:c0:a0
mtu: 1500
nameservers:
addresses: [8.8.8.8]
search:
- maas
parameters:
forward-delay: 15
stp: false
This way the gateway address is directly the address of the bridge. Then, when the VM tries to reach the default route it reach directly the bridge. (I don’t know if I am using the right terms here, more explanation on this are welcome).
Now there is still the internet connectivity issue (thought while the apt update/install works fine within the VM, this latter was still not able to perform wget https://streams.canonical.com/..... for example.
It turns out that we need to enable the IP forwarding and configure a NAT on the host to change the source address of the packets. I used the following commands:
export INTERFACE=$(ip route | grep default | cut -d ' ' -f 5)
export IP_ADDRESS=$(ip -4 addr show dev $INTERFACE | grep -oP '(?<=inet\s)\d+(\.\d+){3}')
sudo sed -i 's/#net.ipv4.ip_forward=1/net.ipv4.ip_forward=1/' /etc/sysctl.conf
sudo sysctl -p
sudo iptables -t nat -A POSTROUTING -o $INTERFACE -j SNAT --to $IP_ADDRESS