Regiond Memory Leak and Database Locks in MaaS 3.4.0

Hi @oitgg ,

We get some reports about memory leaks from time to time, but so far I’ve never been able to reproduce them locally. I was investigating (once again) this issue just 3 days ago but still with no luck.

  • regiond.conf is set to use only 2 workers.

The expected behaviour is that you should see 1 “master” process and 2 “child” processes. Hence you should have 3 regiond processes.

  • Both MaaS regions seem to run queries simultaneously on the same database table, causing locks.

This is expected. Many API calls are locking the DB for different reasons (for example, when you acquire a machine).

Is there any chance to share your DB? If not, it would be useful to get a full sos report and all the MAAS logs. If you can’t share them because you can’t redact the confidential information inside, I’d suggest to try to correlate when a new process is spawned (use ps aux | grep regiond to extract when they are spawned) and what was actually happening in your system and in MAAS (check the logs).