Unable to download some boot-kernels from images.maas.io

Users
1

I’m having an issue downloading some boot kernels from images.maas.io. Some seem to be blocked from downloading. This stopped image downloading around 98% in MAAS GUI. Several boot kernels have the problem per each Ubuntu distribution, mostly 12-15MiB files under the 20250221 or so (latest) directory in the images.maas.io. I tested 24.04LTS(amd64), 22.04LTS(amd64), 20.04LTS(amd64) images in MAAS 3.5, 3.4, 3.3

  1. I tried to download one of them via webpage and wget: unsuccessful
sslee777@maas:~$ wget http://images.maas.io/ephemeral-v3/stable/jammy/amd64/20250221/ga-22.04/generic/boot-kernel
--2025-03-04 02:02:40--  http://images.maas.io/ephemeral-v3/stable/jammy/amd64/20250221/ga-22.04/generic/boot-kernel
Resolving images.maas.io (images.maas.io)... 185.125.190.37, 185.125.190.40, 2620:2d:4000:1::17, ...
Connecting to images.maas.io (images.maas.io)|185.125.190.37|:80... connected.
HTTP request sent, awaiting response... 503 Service Unavailable
2025-03-04 02:02:40 ERROR 503: Service Unavailable.
  1. Error message: Service Unavailable.
sslee777@maas:~$ journalctl -fu snap.maas.pebble -t maas-regiond
(skipped)
Mar  3 18:02:36 maas maas-regiond[36101]: temporalio.activity: [warn] Completing activity as failed ({'activity_id': '1', 'activity_type': 'download-bootresourcefile', 'attempt': 8, 'namespace': 'default', 'task_queue': 'region-internal', 'workflow_id': 'bootresource-download:upstream:6b818071e9a6', 'workflow_run_id': 'f9c9c125-5f57-40de-94b6-baab51534e7f', 'workflow_type': 'download-bootresource'})
Mar  3 18:02:36 maas maas-regiond[36101]: Traceback (most recent call last):
Mar  3 18:02:36 maas maas-regiond[36101]:   File "/snap/maas/38185/usr/lib/python3/dist-packages/temporalio/worker/_activity.py", line 439, in _run_activity
Mar  3 18:02:36 maas maas-regiond[36101]:     result = await impl.execute_activity(input)
Mar  3 18:02:36 maas maas-regiond[36101]:   File "/snap/maas/38185/usr/lib/python3/dist-packages/temporalio/worker/_activity.py", line 694, in execute_activity
Mar  3 18:02:36 maas maas-regiond[36101]:     return await input.fn(*input.args)
Mar  3 18:02:36 maas maas-regiond[36101]:   File "/snap/maas/38185/lib/python3.10/site-packages/maasserver/workflow/bootresource.py", line 248, in download_bootresourcefile
Mar  3 18:02:36 maas maas-regiond[36101]:     raise ApplicationError(
Mar  3 18:02:36 maas maas-regiond[36101]: temporalio.exceptions.ApplicationError: ClientResponseError: 503, message='Service Unavailable', url=URL('http://images.maas.io/ephemeral-v3/stable/jammy/amd64/20250221/ga-22.04/generic/boot-kernel')
(skipped)
  1. local image storage: showing incompleted images
sslee777@maas:/var/snap/maas/common/maas/image-storage$ ls -alFh
total 396M
sslee777@maas:~$ ls -alFh /var/snap/maas/common/maas/image-storage
total 2.5G
drwxr-xr-x 3 root root 4.0K Mar  4 00:34 ./
drwxr-xr-x 7 root root 4.0K Mar  4 02:30 ../
-rw-r--r-- 1 root root  12M Mar  4 02:32 044e143.incomplete
-rw-r--r-- 1 root root  91M Mar  4 00:33 1480afe
-rw-r--r-- 1 root root  15M Mar  4 02:32 1b164dc.incomplete
-rw-r--r-- 1 root root 145M Mar  4 00:33 43eb6b5
-rw-r--r-- 1 root root 315K Mar  2 02:01 4467e0c
-rw-r--r-- 1 root root 1.2M Mar  2 02:01 497bc88
-rw-r--r-- 1 root root 426M Mar  2 02:02 52cd933
-rw-r--r-- 1 root root 1.2M Mar  2 02:01 5897d6f
-rw-r--r-- 1 root root 287K Mar  2 02:01 6a8b8d8
-rw-r--r-- 1 root root  12M Mar  4 02:32 6b81807.incomplete
-rw-r--r-- 1 root root 145M Mar  4 00:33 76dc1c1
-rw-r--r-- 1 root root  91M Mar  4 00:33 7d14037
-rw-r--r-- 1 root root  12M Mar  4 02:32 8314d06.incomplete
-rw-r--r-- 1 root root 169M Mar  2 02:02 839859c
-rw-r--r-- 1 root root 145M Mar  4 00:33 84a342c
-rw-r--r-- 1 root root 108M Mar  2 02:01 86bebc5
-rw-r--r-- 1 root root 425M Mar  4 00:34 9093db8
-rw-r--r-- 1 root root 169M Mar  2 02:02 9274b86
2

It works on my end. Maybe it was a temporary issue

3

I contacted IT personnel at my Univ. He said the university firewall blocks it due to HTTP. They don’t block HTTPS URLs.

So I updated the image source menu on the UI from “maas.io” to “custom” and then put the HTTPS URL https://images.maas.io/ephemeral-v3/stable/ It works fine for now.

Suggestion: Some companies or universities may have similar policies regarding HTTP. It would be great if you could update the MAAS code to access images via HTTPS instead of HTTP. Regiond log shows it tried to access via HTTP like below
‘’‘Mar 03 12:25:47 minikube maas-regiond[1075]: temporalio.exceptions.ApplicationError: ClientResponseError: 503, message=‘Service Unavailable’, url=URL(‘http://images.maas.io/ephemeral-v3/stable/jammy/amd64/20250221/ga-22.04/lowlatency/boot-kernel’)’‘’

4

MAAS is using http on purpose to accomodate Bug #1582836 “use http for stream mirror, not https” : Bugs : MAAS