Well, I’ve been using MAAS to build an openstack cluster.
I’ve completed most of the installation, and we are starting to work with some of the fiddly bits of the networking that Openstack does - which creates a lot of network things on each of the control and compute nodes, such as bonds and bridges and the like. Maas restarted this morning after an update, and now the DNS server is very confused, and refuses to start.
I admit I probably shouldn’t have checked that checkbox that I checked when I deployed the hosts to have them report OS changes back to MAAS periodicaly. Now MAAS’s database is full of duplicated subnets and configurations that don’t make much sense to it - mostly duplicates, and it is restarting a few times a second, failing, rebuilding it’s configuration files (with duplicates) and trying start up again, and crashing…
2022-10-07T15:30:19.299429+00:00 XXX maas.service_monitor: [info] Service 'bind9' is not on, it will be started.
2022-10-07T15:30:19.669814+00:00 XXX maas.service_monitor: [info] Service 'bind9' has been started and is 'running'.
2022-10-07T15:30:19.877073+00:00 XXX maas.dns: [error] Reloading BIND failed (is it running?): Command `rndc -c /var/snap/maas/23947/bind/rndc.conf.maas reload` returned non-zero exit status 1:#012rndc: connect failed: 127.0.0.1#954: connection refused
2022-10-07T15:30:20.273355+00:00 XXX maas.service_monitor: [info] Service 'bind9' is not on, it will be started.
2022-10-07T15:30:20.649533+00:00 XXX maas.service_monitor: [error] Service 'bind9' failed to start. Its current state is 'dead' and 'Result: exit-code'.
2022-10-07T15:30:20.861739+00:00 XXX maas.dns: [error] Reloading BIND failed (is it running?): Command `rndc -c /var/snap/maas/23947/bind/rndc.conf.maas reload` returned non-zero exit status 1:#012rndc: connect failed: 127.0.0.1#954: connection refused
2022-10-07T15:30:21.045558+00:00 XXX maas.service_monitor: [error] Service 'bind9' failed to kill:
That is repeating in the maas.log file. The following is repeating in the named.log file:
07-Oct-2022 15:24:27.432 starting BIND 9.16.1-Ubuntu (Stable Release) <id:d497c32>
07-Oct-2022 15:24:27.432 running on Linux x86_64 5.4.0-126-generic #142-Ubuntu SMP Fri Aug 26 12:12:57 UTC 2022
07-Oct-2022 15:24:27.432 built with '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/x86_64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--enable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=/usr' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' '--disable-isc-spnego' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fdebug-prefix-map=/build/bind9-8cOxCx/bind9-9.16.1=. -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
07-Oct-2022 15:24:27.432 running as: named -c /var/snap/maas/23947/bind/named.conf -S 524288 -g
07-Oct-2022 15:24:27.432 compiled by GCC 9.4.0
07-Oct-2022 15:24:27.432 compiled with OpenSSL version: OpenSSL 1.1.1f 31 Mar 2020
07-Oct-2022 15:24:27.432 linked to OpenSSL version: OpenSSL 1.1.1f 31 Mar 2020
07-Oct-2022 15:24:27.432 compiled with libxml2 version: 2.9.10
07-Oct-2022 15:24:27.432 linked to libxml2 version: 20910
07-Oct-2022 15:24:27.432 compiled with json-c version: 0.13.1
07-Oct-2022 15:24:27.432 linked to json-c version: 0.13.1
07-Oct-2022 15:24:27.432 compiled with zlib version: 1.2.11
07-Oct-2022 15:24:27.432 linked to zlib version: 1.2.11
07-Oct-2022 15:24:27.432 ----------------------------------------------------
07-Oct-2022 15:24:27.432 BIND 9 is maintained by Internet Systems Consortium,
07-Oct-2022 15:24:27.432 Inc. (ISC), a non-profit 501(c)(3) public-benefit
07-Oct-2022 15:24:27.432 corporation. Support and training for BIND 9 are
07-Oct-2022 15:24:27.432 available at https://www.isc.org/support
07-Oct-2022 15:24:27.432 ----------------------------------------------------
07-Oct-2022 15:24:27.432 adjusted limit on open files from 524288 to 1048576
07-Oct-2022 15:24:27.432 found 8 CPUs, using 8 worker threads
07-Oct-2022 15:24:27.432 using 8 UDP listeners per interface
07-Oct-2022 15:24:27.448 using up to 524288 sockets
07-Oct-2022 15:24:27.452 loading configuration from '/var/snap/maas/23947/bind/named.conf'
07-Oct-2022 15:24:27.452 /var/snap/maas/current/bind/named.conf.maas:52: zone '131.251.131.in-addr.arpa': already exists previous definition: /var/snap/maas/current/bind/named.conf.maas:36
07-Oct-2022 15:24:27.452 loading configuration: failure
07-Oct-2022 15:24:27.452 exiting (due to fatal error)
If I try to modify any of the files listed in this error message, they are rebuilt every few seconds.
How do I edit / remove the configuration problems from the MAAS database to get it to stop this behavior?
