Dynamically setting kernel options based on hardware information

Doc Requests
1

It would be useful to see a step by step example of how to set tags with custom kernel options for machines based on their hardware details (using X-path, during commissioning?)

2

E.g. (borrowed from @craig-bender):

Automatically implementing kernel options with MAAS

Without MAAS, the above is a post-installation task which requires administrators to edit the GRUB_CMDLINE_LINUX_DEFAULT parameter in /etc/default/grub across all compute nodes, then perform a reboot on each node.

MAAS has a feature called x-path auto tagging which can do all of the above automatically, including detecting the correct PCI ids, deciding whether Intel or AMD IOMMU should be enabled, and automatically configuring the hosts whose hardware matches the tags definition.

X-Path Auto-Tags

Example 1

The MAAS Tag is applied if

  • Intel VT-d is enabled
  • Host contains Tesla V100 PCIe 16GB GPU

The following Device(s) will be reserved for PCI-Passthrogh

  • Only PCI Devices matching 10de:1db4
maas ${MAAS_PROFILE} tags create \

comment="Enable passthrough for Nvidia Tesla V series GPUs on Intel" \

name=gpgpu-tesla-v-i \

definition='//node[@id="cpu:0"]/capabilities/capability/@id = "vmx" and //node[@id="display"]/vendor[contains(.,"NVIDIA")] and //node[@id="display"]/description[contains(.,"3D")] and //node[@id="display"]/product[contains(.,"Tesla V100 PCIe 16GB")]' \

kernel_opts="console=tty0 console=ttyS0,115200n8r nomodeset modprobe.blacklist=nouveau,nvidiafb,snd_hda_intel nouveau.blacklist=1 nouveau.blacklist=1 nouveau.blacklist=1 video=vesafb:off,efifb:off intel_iommu=on rd.driver.pre=pci-stub rd.driver.pre=vfio-pci pci-stub.ids=10de:1db4 vfio-pci.ids=10de:1db4 vfio_iommu_type1.allow_unsafe_interrupts=1 vfio-pci.disable_vga=1"

Example 2

The MAAS tag reads /usr/share/misc/pci.ids for the PCI IDs of all NVIDIA Tesla V Series GPUs. The MAAS Tag is applied if:

  • Host has Intel VT-d enabled
  • Contains any model of an NVIDIA Tesla V Series PCI-based GPU

The following Device(s) will be reserved for PCI-Passthrogh

  • Any PCI Device matching 10de:1db1, 10de:1db3, 10de:1db4, 10de:1db5, 10de:1db6, 10de:1db7
maas ${MAAS_PROFILE} tags create \

comment="Enable passthrough for Nvidia Tesla V series GPUs on Intel" \

name=gpgpu-tesla-v-i \

definition='//node[@id="cpu:0"]/capabilities/capability/@id = "vmx" and //node[@id="display"]/vendor[contains(.,"NVIDIA")] and //node[@id="display"]/description[contains(.,"3D")] and //node[@id="display"]/product[contains(.,"Tesla V")]' \

kernel_opts="console=tty0 console=ttyS0,115200n8r nomodeset modprobe.blacklist=nouveau,nvidiafb,snd_hda_intel nouveau.blacklist=1 nouveau.blacklist=1 nouveau.blacklist=1 video=vesafb:off,efifb:off intel_iommu=on rd.driver.pre=pci-stub pci-stub.ids=$(awk '/Tesla V/{if (/10de/) print $1":"$2;print "10de:"$1}' /usr/share/misc/pci.ids|paste -sd,) rd.driver.pre=vfio-pci vfio-pci.ids=$(awk '/Tesla V/{if (/10de/) print $1":"$2;print "10de:"$1}' /usr/share/misc/pci.ids|paste -sd,) vfio_iommu_type1.allow_unsafe_interrupts=1 vfio-pci.disable_vga=1"
2 Likes