After MAAS updated to 3.6.0-17541-g.561fe1a3e, I started seeing regular segfaults in the dmesg log.
eg:
tcpdump[98700]: segfault at 0 ip 00007b0249dc420d sp 00007ffecfa97790 error 4 in libc.so.6[7b0249c82000+188000] likely on CPU 24 (core 0, socket 1)
and
tcpdump[98860]: segfault at 0 ip 0000718bba0c420d sp 00007ffcff688a70 error 4
This is on multiple machines. I have a PowerEdge R730xd and several Supermicro systems. All show the same symptom. When I revert to 3.5.4-16349-g.4dbbed5f4 they stop.
There also seems to be some issues with creating lxc VMs. They are failing to boot/load the initial image. More on that as I discover, although I will likely have to just reinstall everywhere to get some stability back.
Hi @brianandrus and thanks for reporting your experience upgrading MAAS to 3.6
To better understand what’s going on and help us investigate further, would you mind opening a bug report?
When filing the bug, please fill in the template with the requested details. In addition, we’d appreciate it if you could attach logs collected using journalctl for the time range where the issue occurs:
journalctl -u snap.maas.pebble.service --since "<start time>" --until "<end time>"
Feel free to include any additional logs you think are relevant (e.g., related to tcpdump or LXD failures).
@javier-fs
I have submitted bug 2109816
Hopefully the issue is found and addressed. Meanwhile, I have removed everything and installed 3.5, which seems to be working.
Fortunately this is the home lab, so I can do that!
What kind of outage are you experiencing after you see the error?
The main thing that impacted me was the fact I could not “power on/off” any LXD vms. That may have nothing to do with the error, but that is what prompted me to start looking at logs and such.
That might indeed be something else. In case you are trying to use LXD trusted passwords, please note that they were removed in LXD 6.0 so you can only use tokens
Always used tokens, so that wouldn’t be the issue. Everything seems ok with Maas 3.5, so that is working. I haven’t deployed the rack maas systems with their lxd vms yet, but do not anticipate issues.
Also noticed that channel=latest/stable no longer exists doing ‘snap maas info’. I had to specify 3.6/stable to run the test and gather data for the ticket.
