layer_m.comp's L expansion (KIND 4) wrote a window's block output to shared
memory and ran the per-pixel head after a workgroup barrier, both inside the
windows' loop, so it could not walk two windows per wave (#error for KIND
>= 3). Every window now writes its output first and the head runs once after
the barrier; L's input stage (KIND 3) gathers its features once, before its
windows' loop, the same way. At one window per wave RADV emits the same
machine code for both stages as before. WPW > 2 stays an error: the head and
the gather need the block's 64 pixels on 64 invocations.
compile_m.py builds dec0_l (and dec0_l_planes) with two windows per wave,
which load each weight tile once for both: 96 -> 216 vector registers, 14 ->
7 waves per SIMD. Not under M_EXACT=1, like the network layers (it would gain
7 us there, 516 -> 509); M_DEFS_DEC0L=-DWPW=1 builds the previous layout. The
input stage at two windows per wave is identical too but slower (452 -> 465
us; 144 -> 256 registers, 10 -> 5 waves), so it stays at one.
Every output frame byte-identical (L, HDR and LDR, 1280x720 -> 1440p,
1114x626 -> 1080p, 1920x1080 -> 1440p, the stress inputs) and so is
everything D4R_M_DUMP writes - with the new default, with both stages at one
window and at two, and in the exact build (one window, and the expansion at
two). The windows' outputs swapped in the head's staging changes every frame.
Preset M's shaders are unchanged.
RX 9070 XT, d4r-m L 720p -> 1440p, interleaved: expansion 383 -> 366 us
(1114x626 -> 1080p: 342 -> 326), frame 2205 -> 2191 us.
Signed-off-by: itchyfeetleech <[email protected]>
layer_m.comp's V tiles (VSH, 4 KiB, per wave and head, accessed only as
matrices and only during the attention) had their own shared memory. Shared
memory caps how many waves this GPU runs, and ACO gives a shader the
registers that many waves leave, so the network layers ran at 3 - 6 waves
per SIMD with registers to spare.
VSH now takes the place of a buffer that is dead during the attention:
- the merge rows NB (enc1, enc2, L's input stage), written after it;
- the expansion NEX (dec1, dec2), read before it;
- HO (L's expansion stage), written after it.
One barrier each before the buffer's own use (none was needed while the
buffers were apart). The tube keeps its own 4 KiB.
| layer | shared memory | vector registers | waves per SIMD |
| enc1 | 13 -> 9 KiB | 256 -> 216 | 4 -> 7 |
| enc2 | 17 -> 13 KiB | 256 | 3 -> 4 |
| dec2 | 13 -> 9 KiB | 256 -> 240 | 4 -> 6 |
| dec1 | 10 -> 6 KiB | 240 -> 192 | 6 -> 8 |
| L input | 16 -> 12 KiB | 192 -> 144 | 8 -> 10 |
| L expansion | 13 -> 9 KiB | 192 -> 96 | 8 -> 14 |
Output byte-identical (M and L, six temporal frames at 1280x720 -> 1440p
and 1114x626 -> 1080p). RX 9070 XT, d4r-m 720p -> 1440p, with the previous
commit: M 1583 -> 1566 us (network 967 -> 951), L 2349 -> 2293 us (input
479 -> 453, expansion 401 -> 382). The tube on one window per wave instead
of two (192 registers, 8 waves) is 9 % slower and stays as it is.
Signed-off-by: itchyfeetleech <[email protected]>
layer_m.comp (M / L, M_RELAXED) converts to FP8 with saturatedConvertEXT,
which RADV lowers as a NaN-preserving clamp per element and the conversion.
As in engine/rr/rr_exact.glsl (FINITE), a value that is never infinite can be
written as "an infinity becomes NaN, then saturate", which RADV turns into the
hardware conversion with FP16_OVFL (f2e4m3fn_satfn, Mesa >= 25.2): the same
result for every finite value and NaN.
Only two sources are provably finite and take it (SATF2_ / MK8_F8F):
- the attention's f32 sums of FP8 products (o);
- the normalised tokens (H1N): the norm sums the squares of the very values
it scales, so an infinite channel makes the reciprocal 0 and the product
NaN, never infinite.
The binary16 accumulators (q, the MLP, the expansion, the residual) can
overflow to infinity, or carry an infinite input pixel, and keep the clamp.
Per network shader 352 -> 64 clamps (one 384 -> 160), 3-4 % fewer
instructions. Output byte-identical (M and L, six temporal frames at
1280x720 -> 1440p and 1114x626 -> 1080p). RX 9070 XT, d4r-m 720p -> 1440p:
M unchanged (1579 / 1583 us, its network is not issue-bound), L 2348 ->
2329 us (input 479 -> 471, expansion 400 -> 386).
Signed-off-by: itchyfeetleech <[email protected]>
- layer_m.comp builds relaxed by default (M_RELAXED: f32 accumulators, paired FP8
conversions, no activation table) with two windows per wave sharing weight tiles
(WPW=2); M_EXACT=1 keeps NVIDIA's roundings, bit-identical to the CUDA path
- layer.comp reuses weight tiles across its waves (TPW=2 in the 32-channel layers)
- engine/rr/rr2_w.comp: one wave per window for preset F's dec0_0 and enc0, used by
compile_rr2.py at the new default D4R_RR_RELAXED=3 (build_native_model.sh too);
D4R_RR_FUSE=1 matches the reference's fused multiply-adds, ptx2rr.py --snorm16
- [Engine] Network defaults to vulkan; the HIP network is opt-in
A standalone Vulkan inference engine (engine/) for DLSS presets K and M that
runs from weights extracted from the user's own nvngx_dlss.dll, with d4r's
own input, network and reconstruction shaders or the native HIP network
layers, recorded into the game's command list by the shim. Opt-in
([Engine] Enabled / D4R_ENGINE=1); anything uncovered falls back to the
CUDA/ZLUDA path with the reason logged.
Byte-identical to NGX's CUDA backend under the phase-capable reference
(1706x960 -> 2560x1440, native HIP network, eleven frames, RX 9070 XT):
- K render- and display-resolution motion vectors, HDR and LDR
- K game-supplied exposure and automatic exposure with any pre-exposure
and exposure scale (one residual: effective exposure exactly 2 or 4)
- K and M with the DLSS indicator's inverted-axis parameters
- M with any exposure settings (NGX ignores them for M)
K packages need the current compile_k.py (marker D4RO0003).
Also: HIP shared buffers are pooled across feature recreation (ROCm leaks
destroyed imports), hipnet.bin must match the preset, M-only games do not
load the K model, install_d4r_runtime.sh stages k, k-ldr, m, m-ldr with
hip/, the activation table has a generator (gen_activation_m.py), and
package_release.sh can bundle the engine's shaders (never its weights).
Signed-off-by: woutermans <[email protected]>
itchyfeetleech