[RESOLVED] Upgrade from 2.9.2 to 3.0.0. UI is upgraded, but version still says 2.9.2

[sagor999]

Hi,
trying to upgrade maas from version 2.9.2 to 3.0.0.
Everything seems to be working fine, but version at the bottom still says 2.9.2 and when calling maas version read it outputs "version": "2.9.2"

During upgrade it did properly run db migrations:

Ensuring database is up to date...                                                                  
Operations to perform:                                                                              
  Apply all migrations: auth, contenttypes, maasserver, metadataserver, piston3, sessions, sites    
Running migrations:                                                                                 
  Applying maasserver.0220_nodedevice... OK                                                         
  Applying maasserver.0221_track_lxd_project... OK                                                  
  Applying maasserver.0222_replace_node_creation_type... OK                                         
  Applying maasserver.0223_virtualmachine_blank_project... OK                                       
  Applying maasserver.0224_virtual_machine_disk... OK                                               
  Applying maasserver.0225_drop_rsd_pod... OK                                                       
  Applying maasserver.0226_drop_iscsi_storage... OK                                                 
  Applying maasserver.0227_drop_pod_local_storage... OK                                             
  Applying maasserver.0228_drop_iscsiblockdevice... OK                                              
  Applying maasserver.0229_drop_physicalblockdevice_storage_pool... OK                              
  Applying maasserver.0230_tag_kernel_opts_blank_instead_of_null... OK                              
  Applying maasserver.0231_bmc_version... OK                                                        
  Applying maasserver.0232_drop_controllerinfo_interface_fields... OK                               
  Applying maasserver.0233_drop_switch... OK                                                        
  Applying maasserver.0234_node_register_vmhost... OK                                               
  Applying maasserver.0235_controllerinfo_versions_details... OK                                    
  Applying maasserver.0236_controllerinfo_update_first_reported... OK                               
  Applying maasserver.0237_drop_controller_version_mismatch_notifications... OK                     
  Applying maasserver.0238_disable_boot_architectures... OK                                         
  Applying maasserver.0239_add_iprange_specific_dhcp_snippets... OK                                 
  Applying maasserver.0240_ownerdata_key_fix... OK                                                  
  Applying metadataserver.0025_nodedevice... OK                                                     
  Applying metadataserver.0026_drop_ipaddr_script... OK                                             

Looking at installed packages on our region-controller, everything seems to be of version 3.0.0:

apt list --installed | grep maas

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

maas-cli/focal,now 1:3.0.0-10029-g.986ea3e45-0ubuntu1~20.04.1 all [installed]
maas-common/focal,now 1:3.0.0-10029-g.986ea3e45-0ubuntu1~20.04.1 all [installed]
maas-dhcp/focal,now 1:3.0.0-10029-g.986ea3e45-0ubuntu1~20.04.1 all [installed]
maas-proxy/focal,now 1:3.0.0-10029-g.986ea3e45-0ubuntu1~20.04.1 all [installed]
maas-region-api/focal,now 1:3.0.0-10029-g.986ea3e45-0ubuntu1~20.04.1 all [installed]
python3-django-maas/focal,now 1:3.0.0-10029-g.986ea3e45-0ubuntu1~20.04.1 all [installed,automatic]
python3-maas-client/focal,now 1:3.0.0-10029-g.986ea3e45-0ubuntu1~20.04.1 all [installed,automatic]
python3-maas-provisioningserver/focal,now 1:3.0.0-10029-g.986ea3e45-0ubuntu1~20.04.1 all [installed,automatic]

regiond itself is also updated:

cat /usr/sbin/regiond
#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'maas==3.0.0','console_scripts','regiond'
__requires__ = 'maas==3.0.0'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('maas==3.0.0', 'console_scripts', 'regiond')()
    )

What is very odd is that I can see that UI is definitely new, all packages are of version 3.0.0.
Any insight as to why it still says version 2.9.2 at the bottom of UI would be greatly appreciated.

Do I just need to manually update something in DB maybe?

[sagor999]

I guess there is a minor bug. So I was doing this upgrade on staging environment. I clones postgresql of production instance, and then was running upgrades on region controller using cloned postgresql to verify that everything is working before rolling out update to production maas instance.
Our production instance have multiple rack controllers and region controllers. In my staging env I removed all those controllers, so that all is left is just region controller that is being updated.
But turns out that removing them from controllers list doesn’t actually remove them from db. There were still present in maasserver_controllerinfo table. After deleting them from that table, version is now properly showing 3.0.0