Trying to passthru my RTX 2080 ti from FreeBSD (14) to a Windows 11 vm (it generates a resource conflict and requires additional installation)

[Marietto]

Hello to everyone.

What I would like to understand if why,when I try to passthru my RTX 2080 ti from FreeBSD to Windows 11,it won't do it,causing the error 12. It says that it generates a resource conflict and requires additional installation).

I'm trying to debug the error. Below you see the IOMMU group of my Nvidia Geforce RTX 2080 ti passed in a Windows 11 / bhyve VM and of Windows 11 installed physically. I'm not able to understand if there are inconsistencies.


VM :
 

Group: 30.0.15.1123 - Device: NVIDIA GeForce RTX 2080 Ti (Device ID: PCI\VEN_10DE&DEV_1E04&SUBSYS_250319DA&REV_A1\3&61AAA01&0&48)

Group: 4.39.0.0 - Device: NVIDIA Virtual Audio Device (Wave Extensible) (WDM) (Device ID: ROOT\UNNAMED_DEVICE\0000)

Group: 1.46.831.832 - Device: NVIDIA USB Type-C Port Policy Controller (Device ID: PCI\VEN_10DE&DEV_1AD7&SUBSYS_250319DA&REV_A1\3&61AAA01&0&4B)

Group: 10.0.22000.2360 - Device: Controller host NVIDIA USB 3.10 eXtensible - 1.10 (Microsoft) (Device ID: PCI\VEN_10DE&DEV_1AD6&SUBSYS_250319DA&REV_A1\3&61AAA01&0&4A)

Physical installation :
 

Group: 31.0.15.5222 - Device: NVIDIA GeForce RTX 2080 Ti (Device ID: PCI\VEN_10DE&DEV_1E04&SUBSYS_250319DA&REV_A1\4&94764DC&0&0009)

Group: 4.49.0.0 - Device: NVIDIA Virtual Audio Device (Wave Extensible) (WDM) (Device ID: ROOT\UNNAMED_DEVICE\0000)

Group: 1.50.831.832 - Device: NVIDIA USB Type-C Port Policy Controller (Device ID: PCI\VEN_10DE&DEV_1AD7&SUBSYS_250319DA&REV_A1\4&9476 4DC&0&0309)

Group: 10.0.22000.2360 - Device: NVIDIA USB 3.10 eXtensible Host Controller - 1.10 (Microsoft) (Device ID: PCI\VEN_10DE&DEV_1AD6&SUBSY S_250319DA&REV_A1\4&94764DC&0&0209)

As an experiment I've passed only this device from FreeBSD to WIndows 11 :
 

pptdevs="2/0/0"

I have excluded "2/0/1 ; 2/0/2 ; 2/0/3". I want to show you what means these addresses :
 

02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)

02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)

02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)

02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)

What happened when I launched the vm using these parameters ?
 

bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \

-s 0,hostbridge \

-s 1,ahci-hd,/mnt/$vmdisk0'p2'/bhyve/img/Windows/Windows11.img,bootindex=1 \

-s 9:0,passthru,2/0/0 \ -s 11,hda,play=/dev/dsp,rec=/dev/dsp \

-s 13,virtio-net,tap18 \ -s 29,fbuf,tcp=0.0.0.0:5918,w=1600,h=950,wait \

-s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \

vm0:18 < /dev/null & sleep 2 && vncviewer 0:18

this error :
 

Assertion failed: (!err), function hda_init, file /usr/corvin-src-releng-140/usr.sbin/bhyve/pci_hda.c, line 353.

The error is caused by this parameter :
 

-s 11,hda,play=/dev/dsp,rec=/dev/dsp \

I have excluded it and Windows 11 booted. Now,inside it I still see the error 12 :

The device PCI\VEN_10DE&DEV_1E04&SUBSYS_250319DA&REV_A1\3&61aaa01&0&48 generates a resource conflict and requires additional installation.

If I pass to the vm only -s 9:0,passthru,2/0/0 \ ; it does not work anyway,because AFAIK,MAYBE there is one and only IOMMU group,so I should pass everything ? But if I pass everything,the error 12 is there anyway.

I would like to understand if there is a problem with the IOMMU thing,if there is maybe there is the needing to develop a patch like this one : GitHub - benbaker76/linux-acs-override for FreeBSD.

For an experienced developer should not be so hard to understand which devices conflict. I want to ask some help to understand how to debug the error. If we will be able to understand which devices conflict,maybe I can explain this to the bhyve developer with which I'm in contact and he can develop a patch. So,we will be able to use every modern nvidia gpus inside a Windows VM with bhyve under FreeBSD ! That's a nice goal,isn't it ? So,please ask me more informations that you need to understand the origin of the error.

 

Windows Build/Version : 10.0.22000.2538

[vinifera]

you'd better ask this on linux forums

*cough* https://www.linuxquestions.org/questions/

[UCyborg]

Yeah, people here really don't have a wide vocabulary, maybe try asking at Emulation and virtualization section on FreeBSD forums.

Even ElevenForum would be better for Win11 specifics.

Edited May 11, 2024 by UCyborg