DHCP service was stopped when the network connection between region and racks was disconnected

Users
,
#1

Hi,

[Environment]
I deployed MAAS 3.3 by snap. My MAAS has 1 region + 5 racks. 1 region + 5 racks are working on different network segments.

I set “IP mode” as “DHCP” for my machines. The DHCP server is managed by MAAS.

[Problem]
I stopped network hardwares for maintenance. The network connection between region and racks was disconnected.

As a result, every DHCP services on rack nodes were stopped. I confirmed it by “maas status” command. And deployed machines lost its IP addresses after lease time.

The DHCP services and machine’s IP addresses were fixed after I booted network hardwares.

[Question]
I hope that DHCP services will continue to work even if the network connection between region and racks is disconnected.

Is there any solution?

I think that one of the solutions is “preparing external dhcp on each network segments”. But it seems that the external dhcp is not recommended.

Or, can I extend lease time of DHCP service managed by MAAS?

(I wouldn’t like to set “IP mode” as “Static”. It is hard to manage IP addresses.)

This documentation presupposes that MAAS-managed DHCP is used to enlist and commission machines. Using an external DHCP server for enlistment and commissioning may work, but note that this is not supported. MAAS cannot manage an external DHCP server, nor can it keep leases synchronised when you return a machine to the pool.

#2

This is tricky. The MAAS architecture is designed such that the region controller needs to communicate with rack controllers for various services, including DHCP, and when you say you “stopped network hardware”, how do you know that you didn’t isolate the DHCP server as well? All that said, I’m not sure about the following suggestions…but to address your issue, you might think of:

  1. Redundancy: You might consider setting up a secondary region controller. This would add redundancy to your MAAS setup and help maintain services in case one region controller goes down. However, it would not resolve the issue if the whole network is down. (TBH, I’m not sure anything would help if the whole network is down; stuff’s just…down.)
  2. External DHCP: As you mentioned, setting up an external DHCP server in each network segment is an option. While it’s true that MAAS prefers to manage the DHCP service itself for better integration and management, using an external DHCP is entirely supported and could be a solution here. Just make sure that you configure the next-server for your external DHCP to feed the address of the TFTP boot server to the machine, or it will just be sitting there with an IP, but no NBP to boot from. And, of course, this external DHCP server would have to be on a network segment that you’re not disconnecting for maintenance.
  3. Lease Time Extension: This could be your best bet. You might be able extend the DHCP lease time in MAAS. I think you can do this with DHCP snippets (which are just little bits of DHCP config that get added to the config file). Note: I haven’t actually tried this. Still, even if it works, this would only delay the issue, not solve it, if the network outage lasts longer than the lease duration. Maybe there’s a process you need to design that involves keeping network maintenance outages within the DHCP lease window that you set.
  4. Rethink Network Maintenance Outages: Why are you taking the network down for maintenance? What are you doing, and are you doing it often? Maybe there’s a better way to maintain your networks that won’t take so much offline for so long.

In an ideal scenario, there should be network reliability and stability between your region controller and rack controllers. If this cannot be guaranteed due to maintenance or other factors, the external DHCP server for each segment could be your best option. This will ensure that machines continue to get IP addresses even when the MAAS region controller is unreachable.

– Bill “just the technical writer guy” Wear

#3

Hi,

Thanks a lot!

I consider the solution based on your information.

I have one more question. You wrote the following information on How to manage DHCP (deb/3.1/CLI) .

If MAAS detects an external DHCP server, it will display it on the rack controller’s page, accessible by selecting ‘Controllers’ from the top menu in the web UI.

I configured and launched a dhcp server by isc-dchp-server package on ubuntu. But, the external dhcp information was not available on the Web UI. Is there any required conditions or operations?

I’m using MAAS 3.3.

If there is possibility that this is caused by my environment or settings, I analyze it by my self.

#4

I resolved the above issue by my self.

The IP addresses of all nodes were configured as static at that time. So DHCP packets were not sent by external dhcp server. So my MAAS couldn’t recognize the external dhcp server.

The external dhcp server was available on MAAS when I configured a ip address of one node as dhcp.

#5

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.