I am running on ubuntu 20.04 and latest snap 3.2.1. We are running 2 rack and 2 region controllers. After doing snap refresh the controllers page doesnt get past loading. (screenshot) All the other pages work just fine. Machines, DNS, subnets images…
If I do a reboot of one of the rack controllers I do get an error telling me " One rack controller is not yet connected to the region. Visit the [rack controllers page] for more information."
but controllers page wont go past loading.
maas status shows all processes running
REGION
bind9 RUNNING pid 3018, uptime 2:19:55
http RUNNING pid 3247, uptime 2:19:51
ntp RUNNING pid 3286, uptime 2:19:51
proxy RUNNING pid 4333, uptime 2:19:41
regiond RUNNING pid 3021, uptime 2:19:55
syslog RUNNING pid 3251, uptime 2:19:51
RACK
bind9 RUNNING pid 2431, uptime 0:23:17
dhcpd RUNNING pid 2811, uptime 0:23:11
dhcpd6 STOPPED Not started
http RUNNING pid 14186, uptime 0:00:45
ntp RUNNING pid 2838, uptime 0:23:10
proxy RUNNING pid 14223, uptime 0:00:45
rackd RUNNING pid 2434, uptime 0:23:17
syslog RUNNING pid 14222, uptime 0:00:45
I did find this error in the log :/var/snap/maas/common/log/http/error.log
2022/08/04 05:16:22 [crit] 3025#3025: *13 connect() to unix:/var/snap/maas/23093/maas-regiond-webapp.sock failed (2: No such file or directory) while connecting to upstream, client: 172.30.0.34, server: , request: “GET /MAAS/rpc/ HTTP/1.1”, upstream: “http://unix:/var/snap/maas/23093/maas-regiond-webapp.sock:/MAAS/rpc/”, host: “172.30.0.30”
2022/08/04 05:16:22 [crit] 3025#3025: *13 connect() to unix:/var/snap/maas/23093/maas-regiond-webapp.sock failed (2: No such file or directory) while connecting to upstream, client: 172.30.0.34, server: , request: “GET /MAAS/rpc/ HTTP/1.1”, upstream: "http://unix:/var/snap/maas/23093/maas-regiond-webapp.sock
it looks like its missing folders?
Update: I think I have found the stack call error in :/var/snap/maas/common/log/regiond.log
when I refresh the webpage, this entry appears
2022-08-04 05:46:40 regiond: [info] 127.0.0.1 GET /MAAS/accounts/login/ HTTP/1.1 --> 200 OK (referrer: http://sv5-maas.avahc.com/MAAS/r/controllers; agent: Mozilla/5.0 (Windows NT
10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36 Edg/103.0.1264.62)
2022-08-04 05:46:40 maasserver.websockets.protocol: [critical] Error on request (1) controller.list: None
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/snap/maas/23093/lib/python3.8/site-packages/provisioningserver/utils/twisted.py", line 821, in worker
return target()
File "/snap/maas/23093/usr/lib/python3/dist-packages/twisted/_threads/_threadworker.py", line 46, in work
task()
File "/snap/maas/23093/usr/lib/python3/dist-packages/twisted/_threads/_team.py", line 190, in doWork
task()
--- <exception caught here> ---
File "/snap/maas/23093/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/snap/maas/23093/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/snap/maas/23093/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/snap/maas/23093/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/snap/maas/23093/lib/python3.8/site-packages/provisioningserver/utils/twisted.py", line 856, in callInContext
return func(*args, **kwargs)
File "/snap/maas/23093/lib/python3.8/site-packages/provisioningserver/utils/twisted.py", line 202, in wrapper
result = func(*args, **kwargs)
File "/snap/maas/23093/lib/python3.8/site-packages/maasserver/utils/orm.py", line 756, in call_within_transaction
return func_outside_txn(*args, **kwargs)
File "/snap/maas/23093/lib/python3.8/site-packages/maasserver/utils/orm.py", line 559, in retrier
return func(*args, **kwargs)
File "/usr/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/snap/maas/23093/lib/python3.8/site-packages/maasserver/websockets/base.py", line 418, in prep_user_execute
return self._call_method_track_queries(
File "/snap/maas/23093/lib/python3.8/site-packages/maasserver/websockets/base.py", line 442, in _call_method_track_queries
result = method(params)
File "/snap/maas/23093/lib/python3.8/site-packages/maasserver/websockets/base.py", line 553, in list
return [self.full_dehydrate(obj, for_list=True) for obj in objs]
File "/snap/maas/23093/lib/python3.8/site-packages/maasserver/websockets/base.py", line 553, in <listcomp>
return [self.full_dehydrate(obj, for_list=True) for obj in objs]
File "/snap/maas/23093/lib/python3.8/site-packages/maasserver/websockets/base.py", line 233, in full_dehydrate
return self.dehydrate(obj, data, for_list=for_list)
File "/snap/maas/23093/lib/python3.8/site-packages/maasserver/websockets/handlers/controller.py", line 180, in dehydrate
vlan_counts[self._vlans_ha[vlan_id]] += 1
builtins.KeyError: None
