Hi,
I have a test setup for Maas which I want to use in production.
This setup came after many iterations, every setup before, the nodes had internet access.
The diagram shows a small part. I came to this setup to isolate racks and have smaller sub-nets.
Layer 1 has only 1 25GbE card per regiond, rackd and nodes, switches use vni/vlan to tie everything together. Thus IPMI is sideband on the NIC. I use a vlan id on the IPMI to isolate traffic from the PXE network.
- Since I only have 1 NIC and I keep PXE boot untagged the rackd’s are connected to the regiond over a vlan (105).
- The untagged nic on rackd is in the nodes subnet
- vlan id 4000 is used for IPMI traffic to the rackd
- The MNGT subnet (10.10.10.8) needs to access all rackd node’s IPMI for some management tasks
- I created gateway’s on PFsense to reach 10.10.13.0/24 and 10.11.13.0/24 from 10.10.10.0/24
- 10.10.30.0/26 is Fabric-0 in Maas
- All other networks are Fabric-1, this was automatically done by Maas
Adding nodes in Maas works perfect, just power on (IPMI vlan id is set beforehand) and the appear in Maas, commission and deploy, all fine. However, after deployment I don’t have internet access, nor can the commission scripts reach 10.10.10.8
I expected to have internet access, since networking is not my strong suite I tried to narrow it down with tracepath and ping, it seems that rackd is not forwarding between subnets how I expected it
- Proxy access on the node subnet is on
- Node can ping 10.10.30.13
- Node cannot ping 10.10.30.60 or any other IP on this subnet besides 10.10.30.13
- Rackd (10.10.30.13) can ping public IP’s and 10.10.10.8 fine
- Other way around, 10.10.10.8 can ping Rackd on 10.10.13.254, but not a node
- I tried adding Fabric-1 subnets in a single space, no change
- I tried setting the kernel ip_forward bit to 1 on Rackd, no change
Why is rackd not forwarding the packages?
Thanks!