Relation "maasserver_routable_pairs" does not exist

Users
1

Hello All,

I’m running the latest version of MAAS 3.5, fresh install. Looking at the regiond logs, I see the errors above (missing relation in the database??).

The maas-region migrate command was executed without any errors.

P.S. We are using PostgreSQL version 16.

Any help?

Jul 16 20:22:52 maas-regiond02 regiond[7076]: maasserver.regiondservices.ntp: [critical] Failed to update NTP configuration.
Jul 16 20:22:52 maas-regiond02 regiond[7076]:         Traceback (most recent call last):
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3.10/threading.py", line 953, in run
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             self._target(*self._args, **self._kwargs)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 821, in worker
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return target()
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/twisted/_threads/_threadworker.py", line 47, in work
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             task()
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/twisted/_threads/_team.py", line 182, in doWork
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             task()
Jul 16 20:22:52 maas-regiond02 regiond[7076]:         --- <exception caught here> ---
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 244, in inContext
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             result = inContext.theWork()  # type: ignore[attr-defined]
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 260, in <lambda>
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 117, in callWithContext
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return self.currentContext().callWithContext(ctx, func, *args, **kw)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 82, in callWithContext
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return func(*args, **kw)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 856, in callInContext
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return func(*args, **kwargs)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 203, in wrapper
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             result = func(*args, **kwargs)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 771, in call_within_transaction
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return func_outside_txn(*args, **kwargs)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 574, in retrier
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return func(*args, **kwargs)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3.10/contextlib.py", line 79, in inner
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return func(*args, **kwds)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/maasserver/regiondservices/ntp.py", line 59, in _getConfiguration
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             peers = ntp.get_peers_for(this_region)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/maasserver/ntp.py", line 56, in get_peers_for
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             peer_addresses_map = get_routable_address_map(peer_regions, node)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/maasserver/routablepairs.py", line 89, in get_routable_address_map
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return group_addresses_by_right_node(routable_addrs)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/maasserver/routablepairs.py", line 102, in group_addresses_by_right_node
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             for _, _, right_node, right_ip in addresses:
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/maasserver/routablepairs.py", line 61, in find_addresses_between_nodes
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             cursor.execute(
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 66, in execute
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return executor(sql, params, many, context)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 79, in _execute
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             with self.db.wrap_database_errors:
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/django/db/utils.py", line 90, in __exit__
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             raise dj_exc_value.with_traceback(traceback) from exc_value
Jul 16 20:22:52 maas-regiond02 regiond[7076]:           File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 82, in _execute
Jul 16 20:22:52 maas-regiond02 regiond[7076]:             return self.cursor.execute(sql)
Jul 16 20:22:52 maas-regiond02 regiond[7076]:         django.db.utils.ProgrammingError: relation "maasserver_routable_pairs" does not exist
Jul 16 20:22:52 maas-regiond02 regiond[7076]:         LINE 3:   FROM maasserver_routable_pairs
2

maasserver_routable_pairs is a VIEW that should have been created by the migrate command.

Currently MAAS still recommends and it’s tested with postgresql v14. Updating our dependencies to 24.04 (noble) is still a work in progress.

3

I’ve just tried to connect my local MAAS to a pg16 instance, and it works as expected.

are you running the region on 24.04 (noble)? this is not supported yet, MAAS 3.5 requires 22.04 (jammy).

4

Hi @vasartori

Did you notice any errors during 3.5 installation?

Can you please check if your database is owned by the correct user and that user has enough permissions?

5

@alexsander-souza

Currently MAAS still recommends and it’s tested with postgresql v14. Updating our dependencies to 24.04 (noble) is still a work in progress.

The region servers are running Ubuntu 22.04, while the PostgreSQL servers are running on Noble. Additionally, I have tested PostgreSQL v14 and observed the same behavior.

I’ve just tried to connect my local MAAS to a pg16 instance, and it works as expected.

Cool, I think the PostgreSQL v16 is not the root problem

@troyanov

Did you notice any errors during 3.5 installation?

No, this include no errors with maas-rack migrate

Can you please check if your database is owned by the correct user

pg_catalog.pg_get_userbyid(d.datdba) as "Owner"
FROM pg_catalog.pg_database d
WHERE d.datname = 'maasdb'
ORDER BY 1;
  Name  | Owner 
--------+-------
 maasdb | maas
(1 row)

user and that user has enough permissions?

                             List of roles
 Role name |                         Attributes                         
-----------+------------------------------------------------------------
 maas      | Superuser, Create DB
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS

postgres=#

I have noticed something: when I start a second regiond, these errors start occurring (but I’m not sure about this. I’ll do more tests with only one server).

My regiond.conf (/etc/maas/regiond.conf) is:

database_host: "10.107.71.30"
database_name: "maasdb"
database_pass: "mypassword"
database_port: 5432
database_user: "maas"
maas_url: "http://10.107.71.25"

The maas_url points to an HAProxy that balances between the regiond servers.

In PostgreSQL, looking at the logs, we see this:

2024-07-17 14:24:03.259 UTC [10752] maas@maasdb ERROR:  relation "maasserver_routable_pairs" does not exist at character 69
2024-07-17 14:24:03.259 UTC [10752] maas@maasdb STATEMENT:  SELECT left_node_id, left_ip,
	       right_node_id, right_ip
	  FROM maasserver_routable_pairs
	 WHERE left_node_id IN (1)
	   AND right_node_id IN (2)
	   AND metric < 4
	 ORDER BY metric ASC

The terminal output can be viewed here: maas-region terminal output · GitHub

6

I guess you were referring to maas-region migrate?
This subcommand will run Django migrations, however you should run maas-region dbupgrade
(Which is actually a post-installation hook for maas-region-controller package)

7

Hi @troyanov

I guess you were referring to maas-region migrate?

Yes!!! My fault. :disappointed:

Running maas-region dbupgrade works! And temporal is working too.

Just out of curiosity: Why does this maas-ansible-playbook/roles/maas_region_controller/tasks/install_maas.yaml at main · canonical/maas-ansible-playbook · GitHub use migrate instead of dbupgrade ?

8

It seems to be a bug :frowning:
May I ask you to file an issue? Thanks in advance!

9

@troyanov bug reported: Bug #2073531 “MaaS Ansible - Wrong command to migrate database” : Bugs : MAAS
Also, I sent a PR: Modified the database migration command by vasartori · Pull Request #156 · canonical/maas-ansible-playbook · GitHub

1 Like
10

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.