Restarting snap maas 3.3.2 breaks MaaS DNS

Users
, ,
1

Got a MaaS server (all in one rackd+regiond) which is authoritative for a maas DNS zone on my network.

snap maas restart produces the following logs in /var/snap/maas/common/log/named.log.

21-Apr-2023 23:50:42.158 resolver priming query complete: success
23-Apr-2023 13:02:55.266 no longer listening on 127.0.0.1#53
23-Apr-2023 13:02:55.274 no longer listening on 192.168.10.4#53
23-Apr-2023 13:02:55.274 no longer listening on ::1#53
23-Apr-2023 13:02:55.278 no longer listening on fe80::dea6:32ff:fe45:6199%2#53
23-Apr-2023 13:02:55.282 shutting down
23-Apr-2023 13:02:55.282 stopping command channel on 127.0.0.1#954
23-Apr-2023 13:02:55.394 exiting
23-Apr-2023 13:03:13.965 starting BIND 9.18.12-0ubuntu0.22.04.1-Ubuntu (Extended Support Version) <id:>
23-Apr-2023 13:03:13.965 running on Linux aarch64 5.15.0-1026-raspi #28-Ubuntu SMP PREEMPT Fri Mar 10 14:28:52 UTC 2023
23-Apr-2023 13:03:13.965 built with  '--build=aarch64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/aarch64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/aarch64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--disable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=yes' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' 'build_alias=aarch64-linux-gnu' 'CFLAGS=-g -O2 -ffile-prefix-map=/build/bind9-WDIBdS/bind9-9.18.12=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
23-Apr-2023 13:03:13.965 running as: named -c /var/snap/maas/27110/bind/named.conf -S 524288 -g
23-Apr-2023 13:03:13.965 compiled by GCC 11.3.0
23-Apr-2023 13:03:13.965 compiled with OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
23-Apr-2023 13:03:13.965 linked to OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
23-Apr-2023 13:03:13.965 compiled with libxml2 version: 2.9.13
23-Apr-2023 13:03:13.965 linked to libxml2 version: 20913
23-Apr-2023 13:03:13.965 compiled with json-c version: 0.15
23-Apr-2023 13:03:13.965 linked to json-c version: 0.15
23-Apr-2023 13:03:13.965 compiled with zlib version: 1.2.11
23-Apr-2023 13:03:13.965 linked to zlib version: 1.2.11
23-Apr-2023 13:03:13.965 ----------------------------------------------------
23-Apr-2023 13:03:13.965 BIND 9 is maintained by Internet Systems Consortium,
23-Apr-2023 13:03:13.965 Inc. (ISC), a non-profit 501(c)(3) public-benefit 
23-Apr-2023 13:03:13.965 corporation.  Support and training for BIND 9 are 
23-Apr-2023 13:03:13.965 available at https://www.isc.org/support
23-Apr-2023 13:03:13.965 ----------------------------------------------------
23-Apr-2023 13:03:13.965 found 4 CPUs, using 4 worker threads
23-Apr-2023 13:03:13.965 using 4 UDP listeners per interface
23-Apr-2023 13:03:13.973 DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
23-Apr-2023 13:03:13.973 DS algorithms: SHA-1 SHA-256 SHA-384
23-Apr-2023 13:03:13.973 HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
23-Apr-2023 13:03:13.973 TKEY mode 2 support (Diffie-Hellman): yes
23-Apr-2023 13:03:13.973 TKEY mode 3 support (GSS-API): yes
23-Apr-2023 13:03:13.977 config.c: option 'trust-anchor-telemetry' is experimental and subject to change in the future
23-Apr-2023 13:03:13.977 loading configuration from '/var/snap/maas/27110/bind/named.conf'
23-Apr-2023 13:03:13.981 reading built-in trust anchors from file '/snap/maas/current/etc/bind/bind.keys'
23-Apr-2023 13:03:13.993 looking for GeoIP2 databases in '/usr/share/GeoIP'
23-Apr-2023 13:03:13.993 using default UDP/IPv4 port range: [32768, 60999]
23-Apr-2023 13:03:13.997 using default UDP/IPv6 port range: [32768, 60999]
23-Apr-2023 13:03:13.997 listening on IPv4 interface lo, 127.0.0.1#53
23-Apr-2023 13:03:14.001 listening on IPv4 interface eth0, 192.168.10.4#53
23-Apr-2023 13:03:14.005 IPv6 socket API is incomplete; explicitly binding to each IPv6 address separately
23-Apr-2023 13:03:14.005 listening on IPv6 interface lo, ::1#53
23-Apr-2023 13:03:14.005 listening on IPv6 interface eth0, fe80::dea6:32ff:fe45:6199%2#53
23-Apr-2023 13:03:14.009 generating session key for dynamic DNS
23-Apr-2023 13:03:14.013 sizing zone task pool based on 8 zones
23-Apr-2023 13:03:14.013 none:99: 'max-cache-size 90%' - setting to 3411MB (out of 3790MB)
23-Apr-2023 13:03:14.017 set up managed keys zone for view _default, file 'managed-keys.bind'
23-Apr-2023 13:03:14.025 command channel listening on 127.0.0.1#954
23-Apr-2023 13:03:14.025 not using config file logging statement for logging due to -g option
23-Apr-2023 13:03:14.029 managed-keys-zone: loaded serial 24
23-Apr-2023 13:03:14.029 zone 255.in-addr.arpa/IN: loaded serial 1
23-Apr-2023 13:03:14.029 zone 0.in-addr.arpa/IN: loaded serial 1
23-Apr-2023 13:03:14.029 zone 10.168.192.in-addr.arpa/IN: loading from master file /var/snap/maas/27110/bind/zone.10.168.192.in-addr.arpa failed: file not found
23-Apr-2023 13:03:14.029 zone 10.168.192.in-addr.arpa/IN: not loaded due to errors.
23-Apr-2023 13:03:14.033 zone maas/IN: loading from master file /var/snap/maas/27110/bind/zone.maas failed: file not found
23-Apr-2023 13:03:14.033 zone maas/IN: not loaded due to errors.
23-Apr-2023 13:03:14.033 zone localhost/IN: loaded serial 2
23-Apr-2023 13:03:14.033 zone maas-internal/IN: loading from master file /var/snap/maas/27110/bind/zone.maas-internal failed: file not found
23-Apr-2023 13:03:14.033 zone maas-internal/IN: not loaded due to errors.
23-Apr-2023 13:03:14.033 zone 127.in-addr.arpa/IN: loaded serial 1
23-Apr-2023 13:03:14.033 all zones loaded
23-Apr-2023 13:03:14.033 running
23-Apr-2023 13:03:20.510 received control channel command 'reload'
23-Apr-2023 13:03:20.510 loading configuration from '/var/snap/maas/27110/bind/named.conf'
23-Apr-2023 13:03:20.510 reading built-in trust anchors from file '/snap/maas/current/etc/bind/bind.keys'
23-Apr-2023 13:03:20.510 looking for GeoIP2 databases in '/usr/share/GeoIP'
23-Apr-2023 13:03:20.514 using default UDP/IPv4 port range: [32768, 60999]
23-Apr-2023 13:03:20.514 using default UDP/IPv6 port range: [32768, 60999]
23-Apr-2023 13:03:20.514 sizing zone task pool based on 8 zones
23-Apr-2023 13:03:20.518 none:99: 'max-cache-size 90%' - setting to 3411MB (out of 3790MB)
23-Apr-2023 13:03:20.522 not using config file logging statement for logging due to -g option
23-Apr-2023 13:03:20.542 reloading configuration succeeded
23-Apr-2023 13:03:20.562 zone maas/IN: loaded serial 1621
23-Apr-2023 13:03:20.566 reloading zones succeeded
23-Apr-2023 13:03:20.582 zone 10.168.192.in-addr.arpa/IN: loaded serial 1621
23-Apr-2023 13:03:20.582 zone maas-internal/IN: loaded serial 1621
23-Apr-2023 13:03:20.582 zone maas-internal/IN: sending notifies (serial 1621)
23-Apr-2023 13:03:20.598 all zones loaded
23-Apr-2023 13:03:20.598 running
23-Apr-2023 13:03:31.430 received control channel command 'freeze maas'
23-Apr-2023 13:03:31.430 freezing zone 'maas/IN': success
23-Apr-2023 13:03:31.562 received control channel command 'thaw maas'
23-Apr-2023 13:03:31.566 zone maas/IN: loaded serial 1622
23-Apr-2023 13:03:31.566 thawing zone 'maas/IN': success
23-Apr-2023 13:03:31.686 received control channel command 'freeze maas-internal'
23-Apr-2023 13:03:31.686 freezing zone 'maas-internal/IN': success
23-Apr-2023 13:03:31.842 received control channel command 'thaw maas-internal'
23-Apr-2023 13:03:31.846 thawing zone 'maas-internal/IN': success
23-Apr-2023 13:03:31.846 zone maas-internal/IN: loaded serial 1622
23-Apr-2023 13:03:31.846 zone maas-internal/IN: sending notifies (serial 1622)
23-Apr-2023 13:03:32.098 received control channel command 'freeze 10.168.192.in-addr.arpa'
23-Apr-2023 13:03:32.098 freezing zone '10.168.192.in-addr.arpa/IN': success
23-Apr-2023 13:03:32.266 received control channel command 'thaw 10.168.192.in-addr.arpa'
23-Apr-2023 13:03:32.266 thawing zone '10.168.192.in-addr.arpa/IN': success
23-Apr-2023 13:03:32.266 zone 10.168.192.in-addr.arpa/IN: loaded serial 1622
23-Apr-2023 13:03:32.470 received control channel command 'reload'
23-Apr-2023 13:03:32.470 loading configuration from '/var/snap/maas/27110/bind/named.conf'
23-Apr-2023 13:03:32.470 reading built-in trust anchors from file '/snap/maas/current/etc/bind/bind.keys'
23-Apr-2023 13:03:32.474 looking for GeoIP2 databases in '/usr/share/GeoIP'
23-Apr-2023 13:03:32.474 using default UDP/IPv4 port range: [32768, 60999]
23-Apr-2023 13:03:32.474 using default UDP/IPv6 port range: [32768, 60999]
23-Apr-2023 13:03:32.478 sizing zone task pool based on 8 zones
23-Apr-2023 13:03:32.494 none:99: 'max-cache-size 90%' - setting to 3411MB (out of 3790MB)
23-Apr-2023 13:03:32.498 not using config file logging statement for logging due to -g option
23-Apr-2023 13:03:32.506 reloading configuration succeeded
23-Apr-2023 13:03:32.526 reloading zones succeeded
23-Apr-2023 13:03:32.530 all zones loaded
23-Apr-2023 13:03:32.530 running
23-Apr-2023 13:03:38.025 no longer listening on 127.0.0.1#53
23-Apr-2023 13:03:38.025 no longer listening on 192.168.10.4#53
23-Apr-2023 13:03:38.025 no longer listening on ::1#53
23-Apr-2023 13:03:38.025 no longer listening on fe80::dea6:32ff:fe45:6199%2#53
23-Apr-2023 13:03:38.029 shutting down
23-Apr-2023 13:03:38.029 stopping command channel on 127.0.0.1#954
23-Apr-2023 13:03:38.045 exiting
23-Apr-2023 13:03:52.580 starting BIND 9.18.12-0ubuntu0.22.04.1-Ubuntu (Extended Support Version) <id:>
23-Apr-2023 13:03:52.580 running on Linux aarch64 5.15.0-1026-raspi #28-Ubuntu SMP PREEMPT Fri Mar 10 14:28:52 UTC 2023
23-Apr-2023 13:03:52.580 built with  '--build=aarch64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/aarch64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/aarch64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--disable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=yes' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' 'build_alias=aarch64-linux-gnu' 'CFLAGS=-g -O2 -ffile-prefix-map=/build/bind9-WDIBdS/bind9-9.18.12=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
23-Apr-2023 13:03:52.580 running as: named -c /var/snap/maas/27110/bind/named.conf -S 524288 -g
23-Apr-2023 13:03:52.580 compiled by GCC 11.3.0
23-Apr-2023 13:03:52.580 compiled with OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
23-Apr-2023 13:03:52.580 linked to OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
23-Apr-2023 13:03:52.580 compiled with libxml2 version: 2.9.13
23-Apr-2023 13:03:52.580 linked to libxml2 version: 20913
23-Apr-2023 13:03:52.580 compiled with json-c version: 0.15
23-Apr-2023 13:03:52.580 linked to json-c version: 0.15
23-Apr-2023 13:03:52.580 compiled with zlib version: 1.2.11
23-Apr-2023 13:03:52.580 linked to zlib version: 1.2.11
23-Apr-2023 13:03:52.580 ----------------------------------------------------
23-Apr-2023 13:03:52.580 BIND 9 is maintained by Internet Systems Consortium,
23-Apr-2023 13:03:52.580 Inc. (ISC), a non-profit 501(c)(3) public-benefit 
23-Apr-2023 13:03:52.580 corporation.  Support and training for BIND 9 are 
23-Apr-2023 13:03:52.580 available at https://www.isc.org/support
23-Apr-2023 13:03:52.580 ----------------------------------------------------
23-Apr-2023 13:03:52.580 found 4 CPUs, using 4 worker threads
23-Apr-2023 13:03:52.580 using 4 UDP listeners per interface
23-Apr-2023 13:03:52.588 DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
23-Apr-2023 13:03:52.588 DS algorithms: SHA-1 SHA-256 SHA-384
23-Apr-2023 13:03:52.588 HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
23-Apr-2023 13:03:52.588 TKEY mode 2 support (Diffie-Hellman): yes
23-Apr-2023 13:03:52.588 TKEY mode 3 support (GSS-API): yes
23-Apr-2023 13:03:52.592 config.c: option 'trust-anchor-telemetry' is experimental and subject to change in the future
23-Apr-2023 13:03:52.592 loading configuration from '/var/snap/maas/27110/bind/named.conf'
23-Apr-2023 13:03:52.596 reading built-in trust anchors from file '/snap/maas/current/etc/bind/bind.keys'
23-Apr-2023 13:03:52.596 looking for GeoIP2 databases in '/usr/share/GeoIP'
23-Apr-2023 13:03:52.596 using default UDP/IPv4 port range: [32768, 60999]
23-Apr-2023 13:03:52.596 using default UDP/IPv6 port range: [32768, 60999]
23-Apr-2023 13:03:52.600 listening on IPv4 interface lo, 127.0.0.1#53
23-Apr-2023 13:03:52.600 listening on IPv4 interface eth0, 192.168.10.4#53
23-Apr-2023 13:03:52.600 IPv6 socket API is incomplete; explicitly binding to each IPv6 address separately
23-Apr-2023 13:03:52.600 listening on IPv6 interface lo, ::1#53
23-Apr-2023 13:03:52.604 listening on IPv6 interface eth0, fe80::dea6:32ff:fe45:6199%2#53
23-Apr-2023 13:03:52.604 generating session key for dynamic DNS
23-Apr-2023 13:03:52.608 sizing zone task pool based on 8 zones
23-Apr-2023 13:03:52.608 none:99: 'max-cache-size 90%' - setting to 3411MB (out of 3790MB)
23-Apr-2023 13:03:52.612 set up managed keys zone for view _default, file 'managed-keys.bind'
23-Apr-2023 13:03:52.616 command channel listening on 127.0.0.1#954
23-Apr-2023 13:03:52.616 not using config file logging statement for logging due to -g option
23-Apr-2023 13:03:52.620 managed-keys-zone: loaded serial 24
23-Apr-2023 13:03:52.624 zone 10.168.192.in-addr.arpa/IN: loading from master file /var/snap/maas/27110/bind/zone.10.168.192.in-addr.arpa failed: file not found
23-Apr-2023 13:03:52.624 zone 10.168.192.in-addr.arpa/IN: not loaded due to errors.
23-Apr-2023 13:03:52.624 zone 255.in-addr.arpa/IN: loaded serial 1
23-Apr-2023 13:03:52.624 zone 0.in-addr.arpa/IN: loaded serial 1
23-Apr-2023 13:03:52.624 zone maas/IN: loading from master file /var/snap/maas/27110/bind/zone.maas failed: file not found
23-Apr-2023 13:03:52.624 zone maas/IN: not loaded due to errors.
23-Apr-2023 13:03:52.624 zone 127.in-addr.arpa/IN: loaded serial 1
23-Apr-2023 13:03:52.624 zone maas-internal/IN: loading from master file /var/snap/maas/27110/bind/zone.maas-internal failed: file not found
23-Apr-2023 13:03:52.624 zone maas-internal/IN: not loaded due to errors.
23-Apr-2023 13:03:52.624 zone localhost/IN: loaded serial 2
23-Apr-2023 13:03:52.628 all zones loaded
23-Apr-2023 13:03:52.628 running
23-Apr-2023 13:06:50.934 network unreachable resolving './NS/IN': 2001:7fd::1#53
23-Apr-2023 13:06:50.938 network unreachable resolving './NS/IN': 2001:500:9f::42#53
23-Apr-2023 13:06:50.938 network unreachable resolving './NS/IN': 2001:dc3::35#53
23-Apr-2023 13:06:50.938 network unreachable resolving './NS/IN': 2001:503:c27::2:30#53
23-Apr-2023 13:06:50.938 network unreachable resolving './NS/IN': 2001:500:200::b#53
23-Apr-2023 13:06:50.938 network unreachable resolving './NS/IN': 2001:500:a8::e#53
23-Apr-2023 13:06:50.938 network unreachable resolving './NS/IN': 2001:500:1::53#53
23-Apr-2023 13:06:50.942 network unreachable resolving './NS/IN': 2001:500:2d::d#53
23-Apr-2023 13:06:50.942 network unreachable resolving './NS/IN': 2001:500:2::c#53
23-Apr-2023 13:06:50.942 network unreachable resolving './NS/IN': 2001:500:12::d0d#53
23-Apr-2023 13:06:50.942 network unreachable resolving './NS/IN': 2001:503:ba3e::2:30#53
23-Apr-2023 13:06:50.942 network unreachable resolving './NS/IN': 2001:500:2f::f#53
23-Apr-2023 13:06:50.942 network unreachable resolving './NS/IN': 2001:7fe::53#53
23-Apr-2023 13:06:50.970 resolver priming query complete: success

From the above logs, it looks like bind is restarted twice, where the second restart breaks the DNS resolution :

  • snap maas restart has been executed around 23-Apr-2023 13:02:55.266. As expected, the DNS service is stopped
  • at 23-Apr-2023 13:03:13.965, bind is started and shows missing zone files, which is probably expected: 23-Apr-2023 13:03:14.033 zone maas/IN: loading from master file /var/snap/maas/27110/bind/zone.maas failed: file not found, 23-Apr-2023 13:03:14.033 zone maas-internal/IN: loading from master file /var/snap/maas/27110/bind/zone.maas-internal failed: file not found
  • at 23-Apr-2023 13:03:20.510, bind is reloaded with no errors. IIUC this is part of the process after zone.maas-internal and zone.maas have been created.
  • then, I’m not sure what causes the zone freeze 11 seconds later at 23-Apr-2023 13:03:31.430 which terminate the bind process: 23-Apr-2023 13:03:38.045 exiting
  • bind is started for the second time a few seconds later: 23-Apr-2023 13:03:52.580 starting BIND and shows the very same errors than previously about missing zone files
  • BUT: no further reload occurs, DNS is definitely broken for the maas zone

my current workaround is to restart the entire maas server

ubuntu@maas:~$ file /var/snap/maas/27110/bind/zone.maas failed
/var/snap/maas/27110/bind/zone.maas: cannot open `/var/snap/maas/27110/bind/zone.maas' (No such file or directory)
failed:                              cannot open `failed' (No such file or directory)
ubuntu@maas:~$ sudo snap list
Name      Version                  Rev    Tracking         Publisher   Notes
core      16-2.58.2                14789  latest/stable    canonical✓  core
core18    20230320                 2724   latest/stable    canonical✓  base
core20    20230308                 1856   latest/stable    canonical✓  base
core22    20230325                 611    latest/stable    canonical✓  base
lxd       5.13-cea5ee2             24761  latest/stable/…  canonical✓  -
maas      3.3.2-13177-g.a73a6e2bd  27110  3.3/stable       canonical✓  -
maas-cli  0.6.8                    81     latest/stable    canonical✓  -
ubuntu@maas:~$
ubuntu@maas:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename:	jammy

Any idea how to investigate further?

2

Difference in named.log when MaaS is started along with the server (see workaround above):

23-Apr-2023 14:10:33.504 exiting
23-Apr-2023 14:12:40.288 starting BIND 9.18.12-0ubuntu0.22.04.1-Ubuntu (Extended Support Version) <id:>
23-Apr-2023 14:12:40.288 running on Linux aarch64 5.15.0-1027-raspi #29-Ubuntu SMP PREEMPT Mon Apr 3 10:12:21 UTC 2023
23-Apr-2023 14:12:40.288 built with  '--build=aarch64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/aarch64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/aarch64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--disable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=yes' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' 'build_alias=aarch64-linux-gnu' 'CFLAGS=-g -O2 -ffile-prefix-map=/build/bind9-WDIBdS/bind9-9.18.12=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
23-Apr-2023 14:12:40.288 running as: named -c /var/snap/maas/27110/bind/named.conf -S 524288 -g
23-Apr-2023 14:12:40.288 compiled by GCC 11.3.0
23-Apr-2023 14:12:40.288 compiled with OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
23-Apr-2023 14:12:40.288 linked to OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
23-Apr-2023 14:12:40.288 compiled with libxml2 version: 2.9.13
23-Apr-2023 14:12:40.288 linked to libxml2 version: 20913
23-Apr-2023 14:12:40.288 compiled with json-c version: 0.15
23-Apr-2023 14:12:40.288 linked to json-c version: 0.15
23-Apr-2023 14:12:40.288 compiled with zlib version: 1.2.11
23-Apr-2023 14:12:40.288 linked to zlib version: 1.2.11
23-Apr-2023 14:12:40.288 ----------------------------------------------------
23-Apr-2023 14:12:40.288 BIND 9 is maintained by Internet Systems Consortium,
23-Apr-2023 14:12:40.288 Inc. (ISC), a non-profit 501(c)(3) public-benefit 
23-Apr-2023 14:12:40.288 corporation.  Support and training for BIND 9 are 
23-Apr-2023 14:12:40.288 available at https://www.isc.org/support
23-Apr-2023 14:12:40.288 ----------------------------------------------------
23-Apr-2023 14:12:40.288 found 4 CPUs, using 4 worker threads
23-Apr-2023 14:12:40.288 using 4 UDP listeners per interface
23-Apr-2023 14:12:40.324 DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
23-Apr-2023 14:12:40.328 DS algorithms: SHA-1 SHA-256 SHA-384
23-Apr-2023 14:12:40.328 HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
23-Apr-2023 14:12:40.328 TKEY mode 2 support (Diffie-Hellman): yes
23-Apr-2023 14:12:40.328 TKEY mode 3 support (GSS-API): yes
23-Apr-2023 14:12:40.332 config.c: option 'trust-anchor-telemetry' is experimental and subject to change in the future
23-Apr-2023 14:12:40.336 loading configuration from '/var/snap/maas/27110/bind/named.conf'
23-Apr-2023 14:12:40.340 reading built-in trust anchors from file '/snap/maas/current/etc/bind/bind.keys'
23-Apr-2023 14:12:40.344 looking for GeoIP2 databases in '/usr/share/GeoIP'
23-Apr-2023 14:12:40.348 using default UDP/IPv4 port range: [32768, 60999]
23-Apr-2023 14:12:40.348 using default UDP/IPv6 port range: [32768, 60999]
23-Apr-2023 14:12:40.348 listening on IPv4 interface lo, 127.0.0.1#53
23-Apr-2023 14:12:40.352 listening on IPv4 interface eth0, 192.168.10.4#53
23-Apr-2023 14:12:40.352 IPv6 socket API is incomplete; explicitly binding to each IPv6 address separately
23-Apr-2023 14:12:40.352 listening on IPv6 interface eth0, fe80::dea6:32ff:fe45:6199%2#53
23-Apr-2023 14:12:40.356 generating session key for dynamic DNS
23-Apr-2023 14:12:40.368 sizing zone task pool based on 8 zones
23-Apr-2023 14:12:40.376 none:99: 'max-cache-size 90%' - setting to 3411MB (out of 3790MB)
23-Apr-2023 14:12:40.380 set up managed keys zone for view _default, file 'managed-keys.bind'
23-Apr-2023 14:12:40.384 command channel listening on 127.0.0.1#954
23-Apr-2023 14:12:40.384 not using config file logging statement for logging due to -g option
23-Apr-2023 14:12:40.404 managed-keys-zone: loaded serial 24
23-Apr-2023 14:12:40.404 zone localhost/IN: loaded serial 2
23-Apr-2023 14:12:40.408 zone 127.in-addr.arpa/IN: loaded serial 1
23-Apr-2023 14:12:40.408 zone 10.168.192.in-addr.arpa/IN: loading from master file /var/snap/maas/27110/bind/zone.10.168.192.in-addr.arpa failed: file not found
23-Apr-2023 14:12:40.408 zone 10.168.192.in-addr.arpa/IN: not loaded due to errors.
23-Apr-2023 14:12:40.408 zone maas/IN: loading from master file /var/snap/maas/27110/bind/zone.maas failed: file not found
23-Apr-2023 14:12:40.408 zone maas/IN: not loaded due to errors.
23-Apr-2023 14:12:40.408 zone maas-internal/IN: loading from master file /var/snap/maas/27110/bind/zone.maas-internal failed: file not found
23-Apr-2023 14:12:40.408 zone maas-internal/IN: not loaded due to errors.
23-Apr-2023 14:12:40.408 zone 0.in-addr.arpa/IN: loaded serial 1
23-Apr-2023 14:12:40.408 zone 255.in-addr.arpa/IN: loaded serial 1
23-Apr-2023 14:12:40.412 all zones loaded
23-Apr-2023 14:12:40.412 running

23-Apr-2023 14:12:47.846 received control channel command 'reload'
23-Apr-2023 14:12:47.846 loading configuration from '/var/snap/maas/27110/bind/named.conf'
23-Apr-2023 14:12:47.846 reading built-in trust anchors from file '/snap/maas/current/etc/bind/bind.keys'
23-Apr-2023 14:12:47.850 looking for GeoIP2 databases in '/usr/share/GeoIP'
23-Apr-2023 14:12:47.850 using default UDP/IPv4 port range: [32768, 60999]
23-Apr-2023 14:12:47.850 using default UDP/IPv6 port range: [32768, 60999]
23-Apr-2023 14:12:47.854 sizing zone task pool based on 8 zones
23-Apr-2023 14:12:47.858 none:99: 'max-cache-size 90%' - setting to 3411MB (out of 3790MB)
23-Apr-2023 14:12:47.866 not using config file logging statement for logging due to -g option
23-Apr-2023 14:12:47.870 reloading configuration succeeded
23-Apr-2023 14:12:47.890 zone 10.168.192.in-addr.arpa/IN: loaded serial 1634
23-Apr-2023 14:12:47.894 zone maas/IN: loaded serial 1634
23-Apr-2023 14:12:47.898 reloading zones succeeded
23-Apr-2023 14:12:47.902 zone maas-internal/IN: loaded serial 1634
23-Apr-2023 14:12:47.902 zone maas-internal/IN: sending notifies (serial 1634)
23-Apr-2023 14:12:47.910 all zones loaded
23-Apr-2023 14:12:47.914 running
23-Apr-2023 14:12:50.426 received control channel command 'freeze maas'
23-Apr-2023 14:12:50.426 freezing zone 'maas/IN': success
23-Apr-2023 14:12:50.570 received control channel command 'thaw maas'
23-Apr-2023 14:12:50.570 thawing zone 'maas/IN': success
23-Apr-2023 14:12:50.570 zone maas/IN: loaded serial 1637
23-Apr-2023 14:12:50.686 received control channel command 'freeze maas-internal'
23-Apr-2023 14:12:50.690 freezing zone 'maas-internal/IN': success
23-Apr-2023 14:12:51.102 received control channel command 'thaw maas-internal'
23-Apr-2023 14:12:51.106 zone maas-internal/IN: loaded serial 1637
23-Apr-2023 14:12:51.106 thawing zone 'maas-internal/IN': success
23-Apr-2023 14:12:51.198 received control channel command 'freeze 10.168.192.in-addr.arpa'
23-Apr-2023 14:12:51.198 freezing zone '10.168.192.in-addr.arpa/IN': success
23-Apr-2023 14:12:51.302 received control channel command 'thaw 10.168.192.in-addr.arpa'
23-Apr-2023 14:12:51.306 thawing zone '10.168.192.in-addr.arpa/IN': success
23-Apr-2023 14:12:51.306 zone 10.168.192.in-addr.arpa/IN: loaded serial 1637
23-Apr-2023 14:12:51.422 received control channel command 'reload'
23-Apr-2023 14:12:51.422 loading configuration from '/var/snap/maas/27110/bind/named.conf'
23-Apr-2023 14:12:51.426 reading built-in trust anchors from file '/snap/maas/current/etc/bind/bind.keys'
23-Apr-2023 14:12:51.426 looking for GeoIP2 databases in '/usr/share/GeoIP'
23-Apr-2023 14:12:51.430 using default UDP/IPv4 port range: [32768, 60999]
23-Apr-2023 14:12:51.430 using default UDP/IPv6 port range: [32768, 60999]
23-Apr-2023 14:12:51.434 sizing zone task pool based on 8 zones
23-Apr-2023 14:12:51.438 none:99: 'max-cache-size 90%' - setting to 3411MB (out of 3790MB)
23-Apr-2023 14:12:51.446 not using config file logging statement for logging due to -g option
23-Apr-2023 14:12:51.450 reloading configuration succeeded
23-Apr-2023 14:12:51.470 reloading zones succeeded
23-Apr-2023 14:12:51.474 all zones loaded
23-Apr-2023 14:12:51.490 running
23-Apr-2023 14:12:52.902 zone maas-internal/IN: sending notifies (serial 1637)
23-Apr-2023 14:13:01.202 received control channel command 'freeze maas'
23-Apr-2023 14:13:01.202 freezing zone 'maas/IN': success
23-Apr-2023 14:13:01.298 received control channel command 'thaw maas'
23-Apr-2023 14:13:01.302 zone maas/IN: loaded serial 1638
23-Apr-2023 14:13:01.302 thawing zone 'maas/IN': success
23-Apr-2023 14:13:01.370 received control channel command 'freeze maas-internal'
23-Apr-2023 14:13:01.370 freezing zone 'maas-internal/IN': success
23-Apr-2023 14:13:01.450 received control channel command 'thaw maas-internal'
23-Apr-2023 14:13:01.454 zone maas-internal/IN: loaded serial 1638
23-Apr-2023 14:13:01.454 zone maas-internal/IN: sending notifies (serial 1638)
23-Apr-2023 14:13:01.454 thawing zone 'maas-internal/IN': success
23-Apr-2023 14:13:01.506 received control channel command 'freeze 10.168.192.in-addr.arpa'
23-Apr-2023 14:13:01.506 freezing zone '10.168.192.in-addr.arpa/IN': success
23-Apr-2023 14:13:01.582 received control channel command 'thaw 10.168.192.in-addr.arpa'
23-Apr-2023 14:13:01.586 zone 10.168.192.in-addr.arpa/IN: loaded serial 1638
23-Apr-2023 14:13:01.586 thawing zone '10.168.192.in-addr.arpa/IN': success
23-Apr-2023 14:13:01.694 received control channel command 'reload'
23-Apr-2023 14:13:01.694 loading configuration from '/var/snap/maas/27110/bind/named.conf'
23-Apr-2023 14:13:01.694 reading built-in trust anchors from file '/snap/maas/current/etc/bind/bind.keys'
23-Apr-2023 14:13:01.694 looking for GeoIP2 databases in '/usr/share/GeoIP'
23-Apr-2023 14:13:01.698 using default UDP/IPv4 port range: [32768, 60999]
23-Apr-2023 14:13:01.698 using default UDP/IPv6 port range: [32768, 60999]
23-Apr-2023 14:13:01.698 sizing zone task pool based on 8 zones
23-Apr-2023 14:13:01.702 none:99: 'max-cache-size 90%' - setting to 3411MB (out of 3790MB)
23-Apr-2023 14:13:01.706 not using config file logging statement for logging due to -g option
23-Apr-2023 14:13:01.706 reloading configuration succeeded
23-Apr-2023 14:13:01.726 reloading zones succeeded
23-Apr-2023 14:13:01.734 all zones loaded
23-Apr-2023 14:13:01.734 running

In this situation, the Zone freeze (executed at 23-Apr-2023 14:13:01.202) does not end up with an exit of the bind process

3

hi @jeanfabrice, could you please file a bug attaching full bind9 logs and regiond/rackd logs as well from around the time when the bug happened?

4

Thanks! Bug created: https://bugs.launchpad.net/maas/+bug/2017684

5

fyi, can not reproduce the issue with 3.4.0.
I think we can close the thread