A standalone Vulkan inference engine (engine/) for DLSS presets K and M that
runs from weights extracted from the user's own nvngx_dlss.dll, with d4r's
own input, network and reconstruction shaders or the native HIP network
layers, recorded into the game's command list by the shim. Opt-in
([Engine] Enabled / D4R_ENGINE=1); anything uncovered falls back to the
CUDA/ZLUDA path with the reason logged.
Byte-identical to NGX's CUDA backend under the phase-capable reference
(1706x960 -> 2560x1440, native HIP network, eleven frames, RX 9070 XT):
- K render- and display-resolution motion vectors, HDR and LDR
- K game-supplied exposure and automatic exposure with any pre-exposure
and exposure scale (one residual: effective exposure exactly 2 or 4)
- K and M with the DLSS indicator's inverted-axis parameters
- M with any exposure settings (NGX ignores them for M)
K packages need the current compile_k.py (marker D4RO0003).
Also: HIP shared buffers are pooled across feature recreation (ROCm leaks
destroyed imports), hipnet.bin must match the preset, M-only games do not
load the K model, install_d4r_runtime.sh stages k, k-ldr, m, m-ldr with
hip/, the activation table has a generator (gen_activation_m.py), and
package_release.sh can bundle the engine's shaders (never its weights).
Signed-off-by: woutermans <[email protected]>
NVIDIA's enc1 and enc2 clamp the patch merge's result to ±2π (as f16) before
its e4m3 encoding (max.f16x2 / min.f16x2 with 0fC0C90FDB / 0f40C90FDB after the
merge's last MMA). The native layers did not, in either template or set:
merge_clamp (swin_common.h) now does, at the three sites that encode the merged
tokens (swin_block.h's e4m3-activation and f16 paths, swin_tok.h), and
swin_check.py's model clamps the same way.
Measured on an RX 9070 XT against a run with no native kernels (preset L,
accuracy set, the flags of Ghost of Tsushima, 1280x720 -> 3840x2160, one
frame): native enc1 alone 34 dB and enc2 alone 31 dB from the translated
kernels before, byte-identical after; the whole native set 28 -> 80 dB (the
rest is dec1, 0.09% of the output values). Preset M's fast set against
translated M: 30.8 -> 35.1 dB after 30 frames. The gfx11 path has the same
change and was not run on hardware.
The repo's tests pass (36, 3 skipped).
L's unfolded enc0 and dec0 are still ZLUDA's compile of NVIDIA's PTX and cost
2.95 ms of the preset's 5.86 ms per frame (RX 9070 XT, 1280x720 -> 4K). The
embedding, the patch expand, the Swin block (32 channels, 2 heads, at pixel
resolution) and the patch merge are now native token-lane code; the texture
work around them stays ZLUDA's compile.
enc0l_tail.hip cuts in at the barrier after the feature staging and dec0l_block.hip
before the output code that reads the block's f16 result out of the last MMAs; make_ptx.py
splices either in when D4R_TEX_SRC names it and build.sh selects both for the gfx12
native-FP8 builds. swin_tex.h carries the indexing the two tails share; swin_tok.h gains
the pixel-order token layout (SWIN_TOK_NATURAL), ZLUDA's attention key order
(SWIN_PV_ZLUDA_ORDER) and swin_tok_core, which the encoded tests reuse. swin_common.h
takes the block weight layout (moved out of swin_block.h) and the exact and approximate
reciprocal / mix forms the tails need without the HIP headers (D4R_NO_HIP_RUNTIME).
The f32 sums stay in ZLUDA's order (K slots of the FP8 WMMAs, key order of the f16
attention product) and rsqrt/rcp use the hardware approximations ZLUDA lowers to, so the
harness's final frame is byte-identical to the translated kernels: all four enc0 flag
variants, 720p and 1080p render, 1440p and 4K output, static and moving scenes, 100
frames, and the accuracy set. With M's own sum order about one token in 20,000 differed.
M's layers compile to the same instructions and its frame is byte-identical.
enc0 1.69 -> 0.63 ms and dec0 1.26 -> 0.54 ms at 720p->4K, 2.73 -> 1.16 and 2.28 -> 1.01
at 1080p->4K; the preset 5.83 -> 4.06 and 8.63 -> 5.69 ms. -DD4R_SWIN_K32_MIX rounds the
f16 products earlier for 0.56 and 0.49 ms and is not byte-identical, so it is not built by
default. The repo's tests pass (36, 3 skipped).
- Add docs/in-place-inputs.md: reading the DLSS inputs in place does not work,
because a hardware texture fetch on a HIP texture object over memory imported
from vkd3d-proton returns stale data while plain loads and cuMemcpy2D are
correct; records the measurements, the rule-outs and the reworked-fetch
workaround that turned out not to pay for itself.
- Add tools/hip_vk_texture_stale_repro.cpp and its build script, the native
Vulkan + HIP reproducer that rules out a general ROCm or RADV bug.
- Tag redirected surfaces as R10G10B10A2 ('R3') as well as RGBA16F ('R2'), and
let the native output kernel store the game's R10G10B10A2 texels itself.
- Import linear, exportable game textures (vkd3d-proton patch 0003) into HIP
so the output kernel writes the game's output texture with no game-side copy.
- Add ID3D12DXVKInteropDeviceD4R3 with GetVulkanLinearImageInfo.
- Gate both paths behind D4R_SHIM_OUTPUT_PACKED and D4R_SHIM_INPLACE, and use
them only for split frames of preset M whose output kernel honours the redirect.
- Convert on the host when a recorded packed frame finds the wrong kernel.
- Give the harness an R10G10B10A2 output, and add D4R_SHIM_SPIN_POLL so
frame timing is not quantised to the poll sleep.
- Document the measured removal of the output copies in performance.md.
make_ptx.py turns the 32 cvt.rn.f16x2.e4m3x2 of rrlite_dec0_4x4_folded into calls of d4r_dec_e4m3x2 (tex_common.h) when D4R_TEX_FP8=1: v_cvt_pk_f32_fp8 and two conversions to f16 instead of ZLUDA's integer lowering (about 25 instructions each; the kernel goes from 4058 to 3418 instructions). The conversion equals ZLUDA's e4m3_to_f16_bits for every code except the NaN codes 0x7f / 0xff, checked over all 65536 code pairs on an RX 9070 XT, and the head never stages those codes (it clamps to +-448 before encoding). Harness frames are byte-identical in the accuracy and the fast set, over a 16-frame moving sequence in the accuracy set. dec0 at 2227x1253 render: 0.43 -> 0.38 ms (accuracy), 0.36 -> 0.30 ms (fast). -DD4R_TEX_SOFT_E4M3 gives ZLUDA's lowering; builds without D4R_TEX_FP8 keep the PTX instruction.
On the gfx12 native-FP8 path the enc0 tail and the dec0 head encode their results with v_cvt_pk_fp8_f32 on the f32 copies of the f16 values (e4m3x4_f32 in tex_common.h) instead of ZLUDA's integer e4m3x2. The clamps keep the conversion in range: +-2pi for enc0 (the f16 bound hi[0], same in f32) and +-448 for dec0, both as a median of three, where a NaN gives the lower bound. The codes equal e4m3x2 for every non-NaN f16, checked over all 65536 patterns on an RX 9070 XT; a NaN in dec0 now encodes as -448 instead of the NaN code. Harness frames are byte-identical in both sets (LDR and HDR, motion vectors at render and display resolution). dec0: 0.21 -> 0.15 ms at 1280x720 render and 0.74 -> 0.53 ms at 2560x1440; enc0 changes by 1-2% (its time is ZLUDA's compile of the feature code). -DD4R_TEX_SOFT_E4M3 (D4R_TEX_CFLAGS) keeps e4m3x2.
The remaining two layers (96 channels, four heads) use the same swin_tok.h template: byte-identical on the same recorded launches and harness sequences, enc2 0.22 -> 0.14 ms and dec2 0.20 -> 0.14 ms at 1280x720 render, 0.75 -> 0.47 and 0.69 -> 0.47 ms at 2560x1440. Skipping windows that lie entirely outside the image is also exact but below the measurement noise (at most 2.4% of the windows of every other tube launch), so it is not kept; true16 code generation remains not tried.
swin_tok.h passes the weights as the first WMMA operand and the activations as the second, so each GEMM's encoded result is already the next one's operand and stays in registers (swin_block.h transposes every result through LDS). One wave owns one 4x4 window for the whole block: norms, attention and the MLP need no LDS and no barriers; only the patch expand and the patch merge go through an e4m3 image in LDS. Outputs are bit-identical to swin_block.h (swapped WMMA operand roles, verified on an RX 9070 XT and on recorded launches). mix2 rounds in place (accumulator is addend and destination, no copy per pair). -DSWIN_NO_TOKEN_LANES builds these layers from swin_block.h again; enc2 and dec2 are unchanged. RX 9070 XT accuracy set, 2560x1440 render: whole evaluation 8.56 -> 7.18 ms.
The 2x figure is against the previous native-FP8 layers. With NativeFp8 off
the RX 9070 XT was already at 3.37 ms for 1280x720 -> 3840x2160; the FP8 set
is 2.88 ms there.
The FP8 Swin layers encode with a clamp to +-448 (NaN-propagating packed
minimum/maximum) followed by v_cvt_pk_fp8_f32, in both the fast and the
accuracy build: equal to half_to_e4m3 for every non-NaN input, NaN encodes
as 0xff. The accuracy build rounds each k32 step with v_fma_mix (K32_MIX).
On gfx12 the accuracy builds of M's post and downsample kernels are wave64
with denormal handling kept.
RX 9070 XT, 1280x720 -> 3840x2160, D3D12 harness: the accuracy set goes
from 3.54 ms to 3.05 ms per evaluation (3.01 -> 2.59 ms under load). Its
Swin layers still equal swin_model.py on sampled blocks and its image is
byte-identical to the previous accuracy set over 8 temporal frames.
On the gfx12 native-FP8 path the Swin layers kept GEMM activations as f16
in LDS and encoded them to e4m3 again at every operand load. They are now
encoded once where they are produced, with v_cvt_pk_fp8_f32, and stored as
bytes (SWIN_A8); the patch merge reads the stored output codes. The M
downsample kernels are built as wave64 on gfx12.
RX 9070 XT, 1280x720 -> 3840x2160, D3D12 harness, DLSS 310.7: whole
evaluation 5.9 ms -> 2.8 ms of GPU time with the GPU otherwise idle, 2.4 ms
at 80-98% GPU utilisation. Harness output is byte-identical to the previous
kernels; the accuracy build matches swin_model.py on sampled blocks.
Builds without D4R_FP8_WMMA (RDNA3, and RDNA4 with NativeFp8 off) compile to
the same instructions as before.
woutermans