HPC with MAAS LXD

[crfigueroa]

Hi everyone!

I’ll be brief, I have a home lab with a MAAS Rack and Controller cluster and 3 nodes, all physical.
Is there the possibility of making an HPC cluster with MAAS and other solutions?
The objective is to create an instance in LXD and that it uses the resources of the 3 nodes, I mean CPU, RAM and Storage.

I would really appreciate any tutorial or any help.

I remain attentive, greetings!

[billwear]

Creating an HPC (High Performance Computing) cluster with MAAS is definitely possible, and – pardon the very shameless cross-recommendations – you can also use other tools such as LXD and Juju to make it waaay easier. Still, creating a single LXD instance that spans multiple nodes with shared CPU and RAM resources is not straightforward. LXD typically operates at a single host level. This means that each LXD container runs on a single host, and does not natively support spanning a single container across multiple hosts.

For HPC tasks, it’s common to use a batch job system like SLURM or a distributed computing framework like Apache Hadoop or Spark. These systems are designed to break a problem into smaller pieces that can be processed in parallel.

If you want to set up an HPC cluster using MAAS, you might consider the following steps:

1. MAAS setup: First ensure that all your nodes are enlisted and commissioned in MAAS. You should be able to see them in the “Machines” tab of the MAAS dashboard.
2. LXD setup: Install and configure LXD on each of the nodes. You can use MAAS to deploy a basic Ubuntu system on each node, then connect to each system to install LXD.
3. Juju setup: Juju can help you orchestrate complex services across your nodes. Install and configure Juju to work with your MAAS and LXD setups.
4. HPC software setup: Choose the appropriate HPC software for your needs (like SLURM, Hadoop, or Spark) and use Juju to deploy it across your nodes.

Here are some useful resources that can help you:

• MAAS documentation
• LXD documentation
• Juju documentation

Remember, HPC isn’t a cakewalk. Be sure to think through your setup really carefully, and RTFM (read the full manuals). Sorry, I’m the MAAS doc person, so yet another shameless plug. Anyway, good luck.

[crfigueroa]

@billwear I thank you very much, you clarified everything for me, in fact I am reading the documentation of MAAS, JUJU, LXD and you are right, it is not to arrive and create a cluster, in fact I have been trying to learn in my spare time for years.

Well I will continue until I achieve it.

A hug and thank you!!

[billwear]

no problem. hugs accepted! good luck, and come back if you have more struggles.