Support for Mellanox 100G in MAAS


#1

I have tried scouring the web for this information, but I’m wondering if MAAS supports out of the box Mellanox 100G cards/interfaces?

I can’t seem to get the Mellanox cards configured using netplan manually and I’m wondering if netplan doesnt support Mellanox, then I’m thinking MAAS wont be able to support it?


#2

Hi @r9labs, not sure if it helps or how things work in the latest MAAS releases, but do you install any Mellanox drivers during enlist/commission/install steps (see Custom node setup docs)? I got a script that installs Mellanox drivers on 16.04 during commissioning from @jamesbeedy so he might know some more.


#3

I don’t currently, but thats something I can look into. It looks like I’ll most likely have to configure a seperate network after the provisioning has occurred.


#4

This is interesting I’d really like to know what you find.
Question though: If you go ahead and deploy Ubuntu 18.04 to the machine with the ConnectX adapter, does it natively find it, or do you need to compile and install and register the ConnectX modules before it turns up in lspci?


#5

Here is the script for installing ConnectX drivers for Xenial from @jamesbeedy in case that helps: https://paste.ubuntu.com/p/nN662VbY8F/


#6

Thanks for that example. I had forgotten about script metadata and didn’t realise that tar.gz files could be downloaded as packages.

I use the following script for my deployments. It’s written so that I can use the same script for any version of Ubuntu supported by Mellanox. The installation itself is a bit custom because I have Infiniband adapters but I have to force then into ethernet mode because MAAS doesn’t seem to inderstand Infiniband. I have to admit that I’m also running a slightly ghetto network configuration.

https://gist.github.com/lparkes/3b9e4e6e772dd1635eec50cc284b7b3a


#7

After much trial and tribulation, we found out that the cards ship by default with the infiniband protocol enabled (IB). We (MAAS) needs it to be configured as ETH protocol. So in order to do that automatically on comissioning, you should download the latest mellanox MFT tool: http://www.mellanox.com/page/management_tools

and then use this as a commissioning script, named 00-maas-00-mftinstall.sh:
https://paste.ubuntu.com/p/NqMvkHRgNT/

you’ll have to host the .tgz file somewhere and update the script to download it from whatever URL you use.

This script will install the Mellanox mangement tools, and then set the firmware on your card to be ETH. After the machine commissions, you’ll be able to configure your mellanox 100G cards in MAAS.