Canonical maas upload boot resource API gives 413 Request entity is too large from postman

prantiknag · 6 September 2024 19:30

unable to upload OS with boot resources api.
Getting 413 error.
Tried to modify nginx.conf file to increase client_max_body parameter but unable to edit that nginx.conf file.

Curl command used—
curl --location ‘http://localhost:5240/MAAS/api/2.0/boot-resources/’
–header ‘Content-Type: application/json’
–header ‘Accept: application/json’
–header ‘Authorization: OAuth oauth_consumer_key=“xLjURY47cRy6Fmx7Rc”,oauth_token=“nsGBtwT5bLTKbH2M25”,oauth_signature_method=“PLAINTEXT”,oauth_timestamp=“1725524095”,oauth_nonce=“ETy9fH90iS9”,oauth_version=“1.0”,oauth_signature=“%266NLF3hyh2jkj84skLZdADuvVmXCJU44y”’
–form ‘content@=@“/home/images/vmware-esxi-8.0U3_V1.0.dd.gz”’
–form ‘name=“esxi/8.0u3”’
–form ‘architecture=“amd64/generic”’
–form ‘filetype=“ddgz”’
–form ‘title=“VMware ESXi 8.0U3-2”’
–form ‘size=“758923583”’
–form ‘sha256=“3af8c42dbd3b9833c17aaf37e2c32d952b33148e1f7d2c8e87c69c81aad12d2b”’

Response: status : 413

r00ta · 7 September 2024 13:18

See How to increase client_max_body in nginx.conf? - #2 by r00ta

prantiknag · 9 September 2024 05:57

Not able to increase client_max_body in nginx.conf from canonical maas environment which is installed by snap.

r00ta · 9 September 2024 06:09

You have to upload the image in chunks

prantiknag · 9 September 2024 06:37

I have already shared the curl command to upload boot resources which takes image as a chunk and getting status of 413 that entity is too large.

what config changes needs to from canonical maas environment?
or do i need to modify the curl command which will work for custom image upload as boot resource in canonical maas?

r00ta · 9 September 2024 06:57

Well the content type is actually wrong and your command is not uploading the image in chunks. See maas/src/maascli/actions/boot_resources_create.py at master · canonical/maas · GitHub for a reference implementation

prantiknag · 9 September 2024 07:12

As per your suggestion invoked the curl command with modified content type as octet-stream but getting same error 413 entity is too large.

curl --location ‘http://localhost:5240/MAAS/api/2.0/boot-resources/’
–header ‘Content-Type: application/octet-stream’
–header ‘Accept: application/json’
–header ‘Authorization: OAuth oauth_consumer_key=“xLjURY47cRy6Fmx7Rc”,oauth_token=“nsGBtwT5bLTKbH2M25”,oauth_signature_method=“PLAINTEXT”,oauth_timestamp=“1725865789”,oauth_nonce=“N4oeVPDw9c2”,oauth_version=“1.0”,oauth_signature=“%266NLF3hyh2jkj84skLZdADuvVmXCJU44y”’
–form ‘content=@“/home/images/vmware-esxi-8.0U3_V1.0.dd.gz”’
–form ‘name=“esxi/8.0u3-1x”’
–form ‘architecture=“amd64/generic”’
–form ‘filetype=“ddgz”’
–form ‘title=“VMware ESXi 8.0U3-1x”’
–form ‘size=“758923583”’
–form ‘sha256=“3af8c42dbd3b9833c17aaf37e2c32d952b33148e1f7d2c8e87c69c81aad12d2b”’
–form ‘base_image=“esxi8.0.U3-1x”’

Output–

413 Request Entity Too Large

nginx/1.18.0 (Ubuntu)

r00ta · 9 September 2024 11:38

still, that’s not a chunked request.

prantiknag · 9 September 2024 14:51

Please provide a example of curl message which accept chunked request.

While invoking boot resources api from postman, in the header it’s passed Accept-Encoding: gzip, deflate, br as a default value as header.

r00ta · 9 September 2024 21:13

You need something like

#!/bin/bash

file_path="/home/ubuntu/rhel9.tar.gz"

CHUNK_SIZE=$((4 * 1024 * 1024))  # 4MB

file_size=$(stat --format="%s" "$file_path")
file_sha256=$(sha256sum "$file_path" | cut -d ' ' -f 1)

response=$(curl -X POST "http://localhost:5240/MAAS/api/2.0/boot-resources/" \
     -H "Authorization: OAuth oauth_version=1.0, oauth_signature_method=PLAINTEXT, oauth_consumer_key=w2Ubk7Dc67XurxtpZf, oauth_token=sx6ut9CBZMKdN3qrfz, oauth_signature=&fvgKG5Kfz6rPB5MUfBsmTbMtSX9gYjrD, oauth_nonce=$(uuidgen), oauth_timestamp=$(date +%s)" \
     -F "size=$file_size" \
     -F "sha256=$file_sha256" \
     -F "name=rhel/9-custom" \
     -F "title=RHEL 9 Custom" \
     -F "architecture=amd64/generic" \
     -F "filetype=tgz")

upload_uri=$(jq -r '.sets | to_entries | sort_by(.key) | reverse | .[0].value.files | to_entries | .[0].value.upload_uri' <<< "$response")

tmp_dir=$(mktemp -d)

split -b $CHUNK_SIZE "$file_path" "$tmp_dir/chunk_"

for chunk in "$tmp_dir"/chunk_*; do
  response=$(curl -X PUT http://localhost:5240$upload_uri \
       -H "Content-Type: application/octet-stream" \
       -H "Content-Length: $(stat --format="%s" "$chunk")" \
       -H "Authorization: OAuth oauth_version=1.0, oauth_signature_method=PLAINTEXT, oauth_consumer_key=w2Ubk7Dc67XurxtpZf, oauth_token=sx6ut9CBZMKdN3qrfz, oauth_signature=&fvgKG5Kfz6rPB5MUfBsmTbMtSX9gYjrD, oauth_nonce=$(uuidgen), oauth_timestamp=$(date +%s)" \
       --data-binary @"$chunk" \
       --write-out "%{http_code}" --silent --output /dev/null)

  if [[ "$response" -ne 200 ]]; then
      echo "Upload failed with status code $response"
      rm -r "$tmp_dir"
      exit 1
  fi
  rm "$chunk"
done

rm -r "$tmp_dir"

echo "Upload complete!"

adjust it according to your credentials and files.

alanbach0 · 5 March 2025 20:14

This was a perfect example! I just slightly modified it to make thins more dynamic:

#!/bin/bash

# Configure your endpoint / API Key Here
MAAS_API_HOST=""
MAAS_API_PATH=""
CONSUMER_KEY=""
CONSUMER_TOKEN=""
SECRET=""

file_path="${1}"
name="${2}"
title="${3}"
arch="${4}"

if [ -z ${arch} ]; then
    arch="amd64/generic"
fi

if [ -z "${file_path}" ] || [ -z "${name}" ] || [ -z "${title}" ]; then
    echo "Usage: maas-api-upload-image.bash <file-path> <name> <title> [arch]"
    exit 1
fi

CHUNK_SIZE=$((4 * 1024 * 1024))  # 4MB Chunks for Multi-Part Uploads

file_size=$(stat --format="%s" "${file_path}")
file_sha256=$(sha256sum "${file_path}" | cut -d ' ' -f 1)

response=$(curl -X POST "${MAAS_API_HOST}/${MAAS_API_PATH}" \
     -H "Authorization: OAuth oauth_version=1.0, oauth_signature_method=PLAINTEXT, oauth_consumer_key=${CONSUMER_KEY}, oauth_token=${CONSUMER_TOKEN}, oauth_signature=&${SECRET}, oauth_nonce=$(uuidgen), oauth_timestamp=$(date +%s)" \
     -F "size=${file_size}" \
     -F "sha256=${file_sha256}" \
     -F "name=${name}" \
     -F "title=${title}" \
     -F "architecture=${arch}" \
     -F "filetype=tgz")

upload_uri=$(jq -r '.sets | to_entries | sort_by(.key) | reverse | .[0].value.files | to_entries | .[0].value.upload_uri' <<< "${response}")

tmp_dir=$(mktemp -d)

split -b ${CHUNK_SIZE} "${file_path}" "${tmp_dir}/chunk_"

for chunk in "${tmp_dir}"/chunk_*; do
  response=$(curl -X PUT ${MAAS_API_HOST}/$upload_uri \
       -H "Content-Type: application/octet-stream" \
       -H "Content-Length: $(stat --format="%s" "${chunk}")" \
       -H "Authorization: OAuth oauth_version=1.0, oauth_signature_method=PLAINTEXT, oauth_consumer_key=${CONSUMER_KEY}, oauth_token=${CONSUMER_TOKEN}, oauth_signature=&${SECRET}, oauth_nonce=$(uuidgen), oauth_timestamp=$(date +%s)" \
       --data-binary @"${chunk}" \
       --write-out "%{http_code}" --silent --output /dev/null)

  if [[ "${response}" -ne 200 ]]; then
      echo "Upload failed with status code ${response}"
      rm -r "${tmp_dir}"
      exit 1
  fi
  rm "${chunk}"
done

rm -r "${tmp_dir}"

echo "Upload complete!"