Proxy vkd3d device extension interfaces

[JoeyDelp]

Updated:
ReShade currently hooks the shared vtable of VKD3D-Proton's ID3D12DeviceExt* interfaces in order to translate ReShade virtual CPU descriptor handles before forwarding CUDA descriptor operations.

Since this modifies the native VKD3D extension object's vtable, the hooks also affect callers that intentionally use the original D3D12 device, for example add-ons using get_native(). Those callers create and pass native descriptor handles, which must not be passed through convert_to_original_cpu_descriptor_handle().

This can result in either an invalid descriptor heap dereference inside ReShade or a native descriptor being translated through an unrelated ReShade descriptor heap before being passed to VKD3D.

This change replaces the shared ID3D12DeviceExt* vtable hooks with an extension proxy returned when the interface is queried through ReShade's D3D12Device proxy.

The proxy forwards the VKD3D extension interfaces through ID3D12DeviceExt5 and only converts descriptor handles for the descriptor-bearing methods:

• GetCudaTextureObject
• GetCudaSurfaceObject
• GetCudaMergedTextureSamplerObject
• GetCudaIndependentDescriptorObject

All other extension methods are forwarded unchanged.

Calls that query an extension interface through ReShade's D3D12 device therefore continue to receive descriptor translation. Calls that obtain the native device first and query the extension interface from that device receive VKD3D's native extension object directly and bypass ReShade.

convert_to_original_cpu_descriptor_handle() remains unchanged.

Testing

Tested with a full add-on Release build on D3D12 through VKD3D-Proton using stock DXVK-NVAPI.

Kingdom Come: Deliverance II was tested with four configurations:

1. Stock ReShade 6.8.0.2155 + RenoDX DLSS5 v4.6: reproduces the original EXCEPTION_ACCESS_VIOLATION during the NR path.
2. ReShade with this proxy change + the same RenoDX v4.6 build: NR initializes and evaluates successfully without the crash.
3. ReShade with this proxy change + the older RenoDX v4.1.5 build that previously worked: NR still initializes and evaluates successfully, confirming that the ReShade descriptor conversion path continues to work.
4. ReShade with this proxy change + current RenoDX v5.0: NR initializes and evaluates successfully.

The proxy implementation was also rebuilt and retested against current ReShade main without regression.

ReShade currently assumes that every `D3D12_CPU_DESCRIPTOR_HANDLE` passed to `convert_to_original_cpu_descriptor_handle()` is one of its own virtual descriptor handles.

This is not necessarily true for private D3D12 extension interfaces. In particular, the VKD3D-Proton CUDA descriptor extension functions may pass native CPU descriptor handles through ReShade's hooks.

A native descriptor can coincidentally contain bits that decode to an existing ReShade descriptor heap index. The current implementation then treats it as a ReShade virtual handle and translates it using the wrong descriptor heap.

I observed two failure modes while debugging this:

• If the apparent heap index does not refer to a live ReShade heap, ReShade can dereference an invalid/null heap entry.
• If it happens to refer to a live heap, the native descriptor can instead be incorrectly translated and the corrupted descriptor is then passed to the underlying D3D12 implementation.

ReShade's virtual CPU descriptor handles already encode enough information to distinguish them more reliably: the heap type/flags are stored in the lower three bits, and the descriptor offset must fall within the referenced heap.

This change therefore only converts a handle when:

1. The encoded heap index refers to a live ReShade heap.
2. The lower metadata bits match that heap's virtual base handle.
3. The encoded descriptor offset is within the descriptor heap.

Otherwise the original handle is returned unchanged.

Testing

Tested with a full add-on Release build on D3D12 through VKD3D-Proton using DXVK-NVAPI.

The reproducer was a ReShade add-on using VKD3D's ID3D12DeviceExt2 CUDA descriptor interfaces. Before this change, enabling the relevant rendering path reliably caused an access violation. With this change and otherwise stock DXVK-NVAPI, the same path initializes and renders correctly.

Existing ReShade virtual descriptor handling also remains functional in the tested workload.

[jettoblack]

Using this PR + jp7677/dxvk-nvapi#391 + a recent master build of vkd3d-proton with VKD3D_CONFIG=descriptor_heap, I finally have Cyberpunk running stable with Reshade + DLSS 4.5 + FG. It took all of those together to make a difference. I can't say it's 100% stable (the game has never been 100% stable, often mod-related issues), but I was able to go from 1-2 minutes before a crash, to well over 1hr+ of gameplay with no Reshade-related crashes. I haven't tried it with DLSS 5.

NVIDIA driver 610.57.04
proton-cachyos-11.0-20260703 + vkd3d-proton @ 35bdee1 + dxvk-nvapi PR 391
cachyos with kernel 7.2.3-1

[flshy1337]

Independent confirmation, from the other side of the same bug: I spent two days tracking this down under Linux/Proton before finding this PR, and both failure modes you describe are exactly what I measured.

How the native handles get in. reshade::hooks::install patches the shared vtable of the vkd3d-proton device object rather than a per-object wrapper, so slots 14/15 of ID3D12DeviceExt2 are hooked for every caller in the process — including a DLSS 5 neural-rendering add-on that takes get_native(), creates its own descriptor heaps on the original device, and never hands ReShade anything. Those handles then go through convert_to_original_cpu_descriptor_handle and come out corrupted, and the fault lands in d3d12_device_vkd3d_ext_GetCudaIndependentDescriptorObject (testb $0x1,(%rax)), which is why this was reported against vkd3d-proton and dxvk-nvapi first (HansKristian-Work/vkd3d-proton#2797, jp7677/dxvk-nvapi#393).

Control experiment. I clobbered one byte of the IID_ID3D12DeviceExt2 constant in ReShade 6.8.0's .rdata, so the QI never matches and slots 14/15 are never hooked. Same bench, same driver, only dxgi.dll differs:

ReShade 6.8.0	result
stock	creating the DLSS feature raised exception 0xC0000005, d3d12core.dll +0x3AEF9C
Ext2 IID clobbered	feature ready at 1920x1080, 2560x1440 and 1260x1370, 12000+ frames

Why a blanket answer is wrong, which is what makes this PR's shape the right one. Four runs, same bench, only the add-on differs:

NR consumer	stock ReShade	Ext2 IID clobbered
renodx-dlssfix 0.2026.903.40	fault on create	ok on create, 12000+ frames
renodx-dlss5 0.2026.828.517	ok on create, fault on evaluate	fault on create

Both sorts of handle reach the same slot: renodx-dlss5 deliberately keeps ReShade's proxy on its D3D12 device, so on create the conversion is required, while on evaluate the same slot receives a native handle and the conversion destroys it. renodx-dlssfix is the mirror image. So neither "always convert" nor "never convert" can work, and per-handle validation is the only thing that can — which is what you implemented.

Your second failure mode (apparent index hits a live heap, corrupted descriptor goes through instead of crashing) matches a class of community reports that read as "it runs fine but the pass does nothing visible", so this PR probably fixes more than the crashes.

Same crash independently reported on: FFXIV, D3D11, RTX 4080 (NIGos/dlss5-bridge#22); LOTRO and Galactic Civilizations IV, D3D11, RTX 4080; DLSS5-Feeder on D3D11; my own bench on an RTX 5070, driver 610.57.04.

I would happily test this on Blackwell and on both add-ons, but there is no build to test: CI here is action_required and no artifact is published, and I am on Linux with no MSVC toolchain. If anyone can attach a dxgi.dll built from this branch, I will run the four-way matrix above against it and post the logs.

[JoeyDelp]

Hi @flshy1337 , please find attached the built x64 Release ReShade64.dll renamed to dxgi.dll for further validation.
dxgi.zip

[crosire]

convert_to_original_cpu_descriptor_handle is a performance critical function and should not be doing parameter verification. It should just never be called with a handle that was not rewritten. So a better solution would need to ensure that any calls from an add-on using native handles cannot reach the hooks, which could e.g. be achieved by proxying the ID3D12DeviceExt* object, rather than hooking the vtable.

[flshy1337]

@JoeyDelp thanks for the build, and sorry for the delay. Ran the four-way matrix against it on the same bench as before (RTX 5070, driver 610.57.04, Proton Experimental, vkd3d-proton, dlss5-bridge synthetic contract with three display-mode changes). Only dxgi.dll differs between the columns.

NR consumer	ReShade 6.8.0 stock	this PR's build
renodx-dlss5 0.2026.828.517	create ok, evaluate faults 0xC0000005, d3d12core.dll +0x3B647C	0 faults, 3600 frames, feature ready at 1920x1080 and 2560x1440
renodx-dlssfix 0.2026.903.40	create faults 0xC0000005, same address	0 faults, 13800 frames, feature ready at 1920x1080, 2560x1440, 1260x1370

So the build fixes both failure modes at once — the consumer that needs its handles converted and the one whose handles must be left alone — which is the result a per-handle decision predicts, and which neither the stock build nor my earlier "never convert" byte patch could give. Logs from all four runs are available if wanted.

On @crosire's point: agreed that the same outcome is reachable without a check in the hot path — proxying the ID3D12DeviceExt* object returned from ReShade's own QueryInterface would mean only handles that came from ReShade ever reach the conversion, and calls made on the native device (which is what these add-ons do via get_native()) would never see the hook at all. The matrix above should serve as the acceptance test for that implementation too: both rows have to come out clean.

[JoeyDelp]

Thanks @crosire and @flshy1337 for the feedback and additional validation. I agree that proxying ID3D12DeviceExt* is the cleaner solution and better preserves the intended invariant for convert_to_original_cpu_descriptor_handle().

I’ll rework the PR in that direction and use the four-way matrix above as the acceptance test. Thanks again for taking the time to test and clarify the constraints.

[JoeyDelp]

Reworked the PR following the feedback from @crosire and @flshy1337.

The shared ID3D12DeviceExt* vtable hooks are now replaced with a proxy returned from ReShade's D3D12Device::QueryInterface. convert_to_original_cpu_descriptor_handle() is back to its original implementation.

Extension calls made through the ReShade device proxy still get descriptor conversion, while callers that obtain the native device first and query the VKD3D extension interfaces bypass ReShade entirely.

I reran the local matrix:

• stock ReShade + RenoDX v4.6: original access violation reproduced
• proxy build + v4.6: NR working, no crash
• proxy build + older v4.1.5: NR working, no regression
• proxy build + current v5.0: NR working

I also rebuilt and retested the change against current upstream main successfully.

@flshy1337, if you have time to rerun your four-way matrix against this version, that would be useful as an independent validation.
ReShade64.zip