Per-layer staged-image engine for the eleven translated denoiser Swin
layers: descriptors (verified offsets per phase), cut recipes that splice an
isolated native phase into NVIDIA's PTX, wave templates, epilogue notes and
GPU probes. The isolated phase-8 splice produces a byte-identical final
4K frame but is slower than the translated body (14.90 vs 6.59 ms) - a
correctness experiment, not a shipping improvement; docs/rr-engine.md
records the design and the next direct-converter win (B operands built
from transposed accumulators).
Ray Reconstruction's eleven DLSS-D Swin layers still run ZLUDA's compile of
NVIDIA's PTX; at 1280x720 -> 3840x2160 on gfx1201 they are 30.2 of a 31.6 ms
frame. The denoiser's high-fidelity output kernel is the first one with a native
replacement: make_ptx.py replaces its whole-texel RGBA16F store
(sust.b.2d.v4.b16.zero) through d4r_sust_v4b16, kernels/build.sh builds the pair
as the `rr` family, and packaging builds it into every target folder including
the RDNA4 FP8 ones.
Both libraries define modules named cuda_dldn_engine_*, with different text, so
one manifest cannot take the union of their hashes: kernel_manifest.py now takes
the denoiser DLL as --rr-dll and lets only its hashes authorize a
cuda_dldn_engine_* binary, and package_release.sh passes it when the denoiser is
bundled.
Verified in the D3D12 harness with Ray Reconstruction at 1280x720 -> 3840x2160:
the output kernel drops from 1.44 to 1.00 ms/frame and the 66 MB raw output is
byte-identical to the run with every kernel translated.
docs/native-kernels.md gains the denoiser family, the per-kernel cost table and
the instruction counts that show why the Swin layers are the remaining work.
L's unfolded enc0 and dec0 are still ZLUDA's compile of NVIDIA's PTX and cost
2.95 ms of the preset's 5.86 ms per frame (RX 9070 XT, 1280x720 -> 4K). The
embedding, the patch expand, the Swin block (32 channels, 2 heads, at pixel
resolution) and the patch merge are now native token-lane code; the texture
work around them stays ZLUDA's compile.
enc0l_tail.hip cuts in at the barrier after the feature staging and dec0l_block.hip
before the output code that reads the block's f16 result out of the last MMAs; make_ptx.py
splices either in when D4R_TEX_SRC names it and build.sh selects both for the gfx12
native-FP8 builds. swin_tex.h carries the indexing the two tails share; swin_tok.h gains
the pixel-order token layout (SWIN_TOK_NATURAL), ZLUDA's attention key order
(SWIN_PV_ZLUDA_ORDER) and swin_tok_core, which the encoded tests reuse. swin_common.h
takes the block weight layout (moved out of swin_block.h) and the exact and approximate
reciprocal / mix forms the tails need without the HIP headers (D4R_NO_HIP_RUNTIME).
The f32 sums stay in ZLUDA's order (K slots of the FP8 WMMAs, key order of the f16
attention product) and rsqrt/rcp use the hardware approximations ZLUDA lowers to, so the
harness's final frame is byte-identical to the translated kernels: all four enc0 flag
variants, 720p and 1080p render, 1440p and 4K output, static and moving scenes, 100
frames, and the accuracy set. With M's own sum order about one token in 20,000 differed.
M's layers compile to the same instructions and its frame is byte-identical.
enc0 1.69 -> 0.63 ms and dec0 1.26 -> 0.54 ms at 720p->4K, 2.73 -> 1.16 and 2.28 -> 1.01
at 1080p->4K; the preset 5.83 -> 4.06 and 8.63 -> 5.69 ms. -DD4R_SWIN_K32_MIX rounds the
f16 products earlier for 0.56 and 0.49 ms and is not byte-identical, so it is not built by
default. The repo's tests pass (36, 3 skipped).
make_ptx.py turns the 32 cvt.rn.f16x2.e4m3x2 of rrlite_dec0_4x4_folded into calls of d4r_dec_e4m3x2 (tex_common.h) when D4R_TEX_FP8=1: v_cvt_pk_f32_fp8 and two conversions to f16 instead of ZLUDA's integer lowering (about 25 instructions each; the kernel goes from 4058 to 3418 instructions). The conversion equals ZLUDA's e4m3_to_f16_bits for every code except the NaN codes 0x7f / 0xff, checked over all 65536 code pairs on an RX 9070 XT, and the head never stages those codes (it clamps to +-448 before encoding). Harness frames are byte-identical in the accuracy and the fast set, over a 16-frame moving sequence in the accuracy set. dec0 at 2227x1253 render: 0.43 -> 0.38 ms (accuracy), 0.36 -> 0.30 ms (fast). -DD4R_TEX_SOFT_E4M3 gives ZLUDA's lowering; builds without D4R_TEX_FP8 keep the PTX instruction.
RDNA4 (experimental; untested on RDNA4 hardware)
- kernels/common/wmma_layout.h picks the WMMA operand/accumulator layout
per target; K (pwin_*) and M (swin_*) index through it, so the same
sources build for gfx11 and gfx12. D4R_WMMA_LAYOUT=12 on gfx11 is a test
shim that runs the gfx12 indexing on gfx11 WMMA.
- M can use native e4m3 FP8 WMMA on gfx12 (D4R_NATIVE_FP8=1). Texture
tails (enc0_tail, dec0_head) get gfx12 f16 and FP8 paths that match
ZLUDA's gfx12 lowering.
- ZLUDA patch 0007: gfx12 WMMA helpers, optional native FP8 MMA
(D4R_ZLUDA_WMMA_FP8_NATIVE), a layout override for testing, and an
architecture argument for d4r_emit. Texture kernels are now compiled
offline per target through d4r_emit (D4R_ZLUDA_EMIT) instead of on the
local GPU.
- New d4r.ini option [Kernels] NativeFp8 (default on), wired through the
shim and d4r_config.py. On gfx12 the bridge serves kernels/<arch>-fp8
when it is on.
- The bridge picks the KFD GPU with the most SIMDs (so an iGPU is not
chosen over a discrete GPU); D4R_GPU_ARCH overrides it.
- kernels/build.sh and package_release.sh build gfx1100-gfx1103 and
gfx1200/gfx1201 (plus -fp8 variants), and D4R_BUNDLE_TEX takes
per-target folders.
Performance (RDNA3; harness byte-identical unless noted)
- ZLUDA patch 0006: m16n8k8 f16 MMAs on WMMA (E; not bit-exact, 52-55 dB
per layer, D4R_ZLUDA_WMMA_K8=0 disables it), per-key weight-image slots
for native prep kernels (d4r_prep_key_at, d4r_prep_key_slots), and
wave64 builds of MMA-free kernels (D4R_ZLUDA_WAVE64).
- M: packed FP8 encoding, conflict-free patch-merge reads, scalar weight
loads in enc1, and weight prep once per weight set for the six enc3
tube blocks.
- K: CU mode for the six non-wide pwin layers.
- Texture kernels: every DLSS flag variant is built natively (24 K output,
8 M post, 4 M downsample, 4 M enc0), not only Townfall's; post and K
output run as wave64.
- M direct output: rrlite_downsample_kernel writes the shim's output
buffer; the shim's default direct-output presets are now 11,13.
Tools
- dump_runner takes the prep grid from d4r_prep_blocks and refuses
texture/surface dumps.
- New pwin_check.py, swin_check.py, model_enc3.py and tail_check.hip.
Docs: README GPU table and RDNA4 notes, building (patches 0002-0007,
d4r_emit), native-kernels (RDNA4 section), performance (new numbers),
packaging README. Version 0.1.2.
woutermans