Upgrading Server RAM (Proxmox No IOMMU detected)

Published on

As it turns out, upgrading RAM isn't quite as simple as I first thought. At least this time around, and it should be more resillient next time.

The Problem

My Dell PowerEdge T430 came with 64GB of RAM, but using Proxmox with many VMs, I found my RAM consumption to be sitting at maximum... This wasn't ideal, but it worked for the most part...

So I obviously shut everything down, opened her up and installed the RAM. All went well, aside from a bit of dust which was nothing I couldn't handle, and away the server went again. However, once booted it up, the worrying sign that my TrueNAS VM hadn't started suddenly became clear...

What was wrong? It turns out the PCIe RAID controller was no longer being passed through to the VM and so the VM couldn't start, complaining with:

no iommu detected please activate it see documentation for further information

Was this the time to start panicking? I thought I'd covered every assumption when it comes to recovery, but this!? I hadn't prepared for it!

I guess it's time to try some fixes, I thought!

First Assumption

I naively followed the initial pieces of information I came across on the internet, because, why not? Of course the internet is correct.

But unfortunately, nothing seemed to work. At least now I do have my server's BIOS up-to-date!

Back to Basics

First things first, I needed to determine which bootloader I was using, it was rather stupid of me to assume it was GRUB... /facepalm Anyway, this was as simple as running:

# efibootmgr -v

which yielded a result containing:

Boot0005* Linux Boot Manager    [...] File(\EFI\systemd\systemd-bootx64.efi)

With this, I could conclude I was in fact not using GRUB, and I was using proxmox-boot-tool and I could run:

# proxmox-boot-tool status

to verify.

For what it's worth, if you see the result containing:

Boot0005* proxmox       [...] File(\EFI\proxmox\grubx64.efi)

Then you are using GRUB in UEFI mode, and a message stating EFI varaibles are not supported, indicates GRUB is being used in BIOS/Legacy mode.

The True Solution

Now I knew what I was actually working with, instead of modifying /etc/default/grub, I needed to modify /etc/kernel/cmdline.

The solution for both is to add intel_iommu=on in my case on the single line, in the case of a GRUB bootloader, appended to the GRUB_CMDLINE_LINUX_DEFAULT value.

Once added, you can run proxmox-boot-tool refresh, or in the case of GRUB, update-grub - these are important steps...

Then you can reboot and fingers crossed everything just works again! It did for me, at least!