I am seeing a weird issue where MAAS is not responding to DHCP request from iDRAC of a Dell Server which is setup using MAAS.
Dell poweredge is up and running and all of a sudden power functionality in MAAS stopped working as ipaddress of iDRAC is not reachable. When I rebooted the server, I can see that iDRAC ipaddress is 0.0.0.0 and DHCP requests are sent to MAAS without any replies back.
As shown below, I can see the DHCP discover message in tcpdump
But when I check DHCP logs as shown below it says DHCP discover is received on VLAN 5004 and no leases present for the same. As shown in previous packet capture I can only see untagged DHCP discover message. This is really weird as vlan 5004 is not even within valid vlan range
# grep dhcpd /var/log/syslog | tail -10
Oct 4 12:08:28 mm431 dhcpd[1600]: DHCPOFFER on 10.10.10.1 to 52:54:00:47:65:e5 via enp7s0
Oct 4 12:08:30 mm431 dhcpd[1600]: DHCPDISCOVER from 52:54:00:47:65:e5 via enp7s0
Oct 4 12:08:30 mm431 dhcpd[1600]: DHCPOFFER on 10.10.10.1 to 52:54:00:47:65:e5 via enp7s0
Oct 4 12:08:34 mm431 dhcpd[1600]: DHCPDISCOVER from 52:54:00:47:65:e5 via enp7s0
Oct 4 12:08:34 mm431 dhcpd[1600]: DHCPOFFER on 10.10.10.1 to 52:54:00:47:65:e5 via enp7s0
Oct 4 12:09:24 mm431 dhcpd[1600]: DHCPDISCOVER from 84:2b:2b:0c:5d:9f via enp7s0: network vlan-5004: no free leases
Oct 4 12:14:06 mm431 dhcpd[1600]: message repeated 14 times: [ DHCPDISCOVER from 84:2b:2b:0c:5d:9f via enp7s0: network vlan-5004: no free leases]
Oct 4 12:15:09 mm431 dhcpd[1600]: DHCPDISCOVER from 84:2b:2b:0c:5d:9f via enp7s0: network vlan-5004: no free leases
Oct 4 12:16:24 mm431 dhcpd[1600]: message repeated 5 times: [ DHCPDISCOVER from 84:2b:2b:0c:5d:9f via enp7s0: network vlan-5004: no free leases]
Oct 4 12:17:27 mm431 dhcpd[1600]: DHCPDISCOVER from 84:2b:2b:0c:5d:9f via enp7s0: network vlan-5004: no free leases
The server is in deployed stage with the latest ubuntu server 22.04 and sitting idle for last 4 days.
We started observing an issue in MAAS UI saying POWER operation is not working as IPMI is not reachable. At this point ping to iDRAC ipaddress is failing.
Since this server is remotely located, we tried power reboot multiple times and the situation has not improved. Yesterday we connected the monitor to the server and performed a power reboot and observed that iDRAC is not receiving any ipaddress from MAAS DHCP.
On further troubleshooting, we observed that DHCP requests from the Server [both iDRAC and Host] are ignored by MAAS. As shown in my previous logs, packet capture on MAAS shows DHCP DISCOVER messages are received from the server in untagged format. But DHCP server is displaying a log saying DHCP-DISCOVER from server is part of network vlan-5004 and it contains no free leases.
Vlan range is valid only from 2-4096 and I am confused why DHCP server says it is vlan-5004.
NOTE: I have tried replacing SWITCH1 with SWITCH2 and it seems to work fine. I see the same packet on MAAS end with both SWITCH1 and SWITCH2 but it works fine with SWITCH2 only.
Well even I was thinking it could be the switch issue but the packet captures were showing untagged frames and this confused me. Anyhow I will try to trouble shoot it at the switch level to confirm the same