But the more relevant part is that since your Win10 install is on a VM, it can’t do shit on the rest of your system, and the GPU access is just there so that it won’t run as slow as shit when gaming, right?

Pretty much

I tried to look it up, and as far as I understood it, it’s a technique that allows a virtual machine to access a physical GPU directly. I guess that means that even if your VM is elsewhere (a server or wherever) it can still use the GPU you have.

So, to get more technical, there's a motherboard technology called IOMMU, which was developed for containing malware that has infected device firmware. What Linux has is a kernel module that allows an IOMMU group to be isolated from the host operating system, and connected up to a virtual machine as if it were real hardware. On an expensive motherboard, you get a different IOMMU group for each PCIe lane, each M.2 socket, each cluster of USB ports, etc. On a cheap one, you get one that for each type of device, maybe the PCIe lanes are divided into two groups.

So the fun part, and why we do this, is that when you have two GPUs, in different IOMMU groups, one can remain on host and allow graphics drivers, desktop environment, etc. to remain loaded, while the other can be connected to the VM and used entirely for gaming (theoretically, if you wanted to you could game on both systems at once). Thankfully, cheap, shit secondary GPUs aren't expensive (was once on a 710, ditched that and its many driver issues for a 1050, and my main remains a 980ti), but setting up the main GPU to switch between proper drivers and "vfio-pci", the drivers that have to be loaded before the passthrough can occur, can be a pain.