d4r/docs/windows-performance.md

Windows RDNA4 measurements

Hardware: RX 9070 XT / gfx1201, Windows 11, display driver 32.0.31041.1004. Runtime: TheRock 10.2.0a20260929, HIP 7.17.26386; ZLUDA b0161a4 with D4R_ZLUDA_WMMA=1, FP8 widening, native FP8 disabled, f16 reference rounding.

The optional profiler uses HIP events on the actual launch stream, including separate measurements of one-time weight preparation. It synchronizes each sampled launch and changes scheduling: these numbers are not application FPS. HIP occupancy predictions are not measured hardware occupancy. The native objects' actual wave32, LDS, VGPR/SGPR and spill counts are available in llvm-readobj --notes and can be attached to the JSON report. Memory bandwidth and WMMA utilization were not measured; no utilization is inferred from timing.

The identical four-frame standalone D3D12/OptiScaler workloads output 512x288. Summing the measured main-kernel event times and dividing by four gives:

Preset	Fully translated, ms/frame	Native FP16-equivalent, ms/frame
K	157.745441	0.684675
M	7.428538	1.566500

Both rows include translated input/output tails. Native overrides cover all 11 K layers and five M layers; M's tube layer executes six times per frame. M native and translated RGB are bit-exact, with max absolute/relative error 0. K native remains bit-exact against its independently validated native control; fully translated K has the previously documented arithmetic differences. All outputs are finite. These are reference-correct codegen settings, not measurements of a relaxed fast-math translated configuration.

M's first profiled Evaluate includes about 0.9 seconds of host initialization; the four-sample CPU mean must not be treated as steady-state latency. Its median Evaluate host completion is 4.855 ms. GPU events and prep phases are reported separately in the raw JSON.

The initial game runner used PowerShell line-oriented output redirection. It heavily throttled verbose stdout/stderr. The runner now concurrently drains raw byte streams using the same approach as the standalone diagnostics.

Silent Hill 2 stage, CPU mean	Initial capture, ms	Raw-byte capture with HIP profiler, ms
Input fence wait	5.641	0.833
Input conversion and CUDA array upload	13.196	0.668
NGX Evaluate host call	25.715	3.350
NGX all-stream completion	1.846	0.033
Output CUDA array download	4.288	0.167
Output conversion and shared fence signal	1.500	0.033

The first run additionally scanned every output (10.142 ms in that throttled capture); the kernel-profiling run disables that scan. Both use a temporary 1280x720 window and current-frame VRAM interop. These are separate menu/game runs, not a controlled whole-game FPS comparison. No synchronization was removed to obtain the improvement. The raw-byte profiling run completed 2345 K frames, with no d4r failure, CPU image copy or previous-frame output.

Reproduce native or translated measurements with locally supplied NVIDIA DLLs at the repository root and the separately built profiling runtime:

.\scripts\windows\profile-windows-rdna4.ps1 -Preset 11
.\scripts\windows\profile-windows-rdna4.ps1 -Preset 11 -Translated
.\scripts\windows\profile-windows-rdna4.ps1 -Preset 13
.\scripts\windows\profile-windows-rdna4.ps1 -Preset 13 -Translated

profile_report.py <results-directory> --output profile.json summarizes raw CPU stages and every kernel's event time, launch geometry, registers, private memory, LDS and predicted blocks per multiprocessor. Add --metadata-directory <directory-of-llvm-readobj-notes> for native wave sizes and compiler spill counts. Reports and proprietary workload inputs remain local.

Native FP8 remains a separate optional experiment; the package uses the validated FP16-equivalent baseline. Its later checks and timings are below.

K host synchronization and asynchronous submission

Current priority is K performance. The user's synchronous 4K reading is 49–51 FPS / 53% GPU utilization, versus approximately 80+ FPS / 100% with FSR4. These readings do not identify a CPU hardware limit or provide a controlled same-scene comparison. M/FP8 optimization is deferred.

ZLUDA candidate a1c506f adds D4R_ZLUDA_PROFILE_API, enabled by the game runner's -ProfileCudaApi. It measures existing API host durations, aggregates per thread and logs every two seconds without HIP events or additional waits. The synchronous control shows five cuCtxSynchronize calls per frame with about 3.9 ms aggregate host time. CUDA launch host overhead is approximately 0.46 ms per frame. Cached copy-list setup is only 0.011 ms median. These host measurements include GPU waits and must not be added to GPU kernel times.

K/4K command submission, CPU median	Synchronous	First async	Async without CUDA mutex
Entire submission, ms	10.512	2.676	0.181
DLSS boundary, ms	10.083	2.604	0.124
Consumer suffix completion, ms	0.350	0.005	0.004

The first async game run completes 9481 finite frames; the user reports 60 FPS / 63% GPU. The mutex-free run crashes during feature recreation after 366 finite frames, so its submission timing is not a successful gameplay benchmark. The standalone burst/recreate reproducer subsequently identifies caller-owned GPU backing freed after ReleaseFeature returns. Waiting for pending D3D12 consumers in ReleaseFeature fixes the reproducer: all twelve sync/async test frames pass, and corresponding RGB outputs are exact. Game stability and FPS must be checked separately on the fixed build.

The fixed game retest on 2026-10-02 completes 16941 K/4K frames, all finite, including feature recreation and color-format reimports, with zero backend failures. The user reports 62 FPS / 63% GPU. Median CPU submission is 0.197 ms, NGX host completion on the worker is 4.400 ms and output validation is 0.380 ms. The 10.162 ms median worker input wait includes queued game rendering. These results establish a stable five-minute test of this fix, not a conclusion about the remaining GPU idle time or FSR4 performance.

-AsyncInterop is opt-in. It queues input/output fences on the submission thread, executes CUDA on a worker and retires owned command memory by GPU completion. Three copy slots bound outstanding ownership. The queue consumes the output of the same frame, with no CPU image copy. The worker's input wait includes its lead behind submitted rendering and is not a CPU submission stall. Neither that duration nor the NGX recording interval measures Present latency. Reconfiguration waits for all consumers; ordinary submission does not hold the CUDA mutex. The experimental runtime currently supports one queue.

.\scripts\windows\test-async-k.ps1

The test generates fresh synchronous controls, submits three frames without intermediate CPU completion, and repeats while releasing/recreating the DLSS feature with work pending. Results include exact RGB comparisons, finite GPU scans, stdout/stderr, exceptions and runtime provenance. To include the local validated output-store optimization, pass its combined native directory with -NativeRoot; proprietary objects remain local.

-ProfileGpuBoundary (requires -AsyncInterop) adds D3D12 timestamps before and after the VRAM input/output copies. Resolved results are read only once the copy slot's existing fence completes; 32 diagnostic bytes per frame are read and no new completion wait is added. external_span_ms spans HIP and host/driver scheduling across the external fence. between_boundaries_ms also includes game rendering, and input_interval_ms is not Present timing. These queries obey the queue's timestamp ordering and add GPU commands; they are disabled for an ordinary FPS run. GPU clock behavior during idle can affect intervals; no hardware occupancy is inferred. Timestamp semantics: Microsoft D3D12 timing.

The timestamp-enabled 4K burst/recreation gate completes twelve frames, all finite; all six async RGB outputs still match their fresh controls exactly. Each async mode emits all three timing records, including the final frame at feature release. Results: test-results/k-async-gpu-boundary-4k. All sixteen CTest gates pass on the rebuilt shim. Reproduce with:

.\scripts\windows\test-async-k.ps1 -ProfileGpuBoundary

Further game coverage (HIP kernel profiling disabled, GPU output checks enabled): M at 1920x1080 completes 4833 frames, all finite. K at 3840x2160 completes 4685 frames, all finite. Neither has a backend failure, CPU image copy or an aged-frame fallback. Their mean interop/NGX CPU stages total about 13.6 and 10.1 ms respectively; these are different scenes, not a K/M comparison.

The no-debugger K/4K run completes 5774 frames, with 5774 finite output scans and 63514 native transformer launches. The user reports 38-42 FPS. Additional CPU measurements exclude replay/publication as the main cause of low FPS:

CPU measurement	Mean, ms	Median, ms
Command split setup	0.284	0.275
State replay within split setup	0.025	0.023
Submitted texture state publication	0.029	0.029
Consumer suffix completion	0.405	0.409
NGX recording interval	27.474	28.466

Recording intervals are between NGX recording calls on the same thread, not DXGI Present times. The interval includes rendering and application scheduling. A second K/4K run with the attached debugger completes 3567 frames without failure, with median recording interval 30.091 ms. The runs include different menu/game states; this does not establish a precise debugger FPS speedup. Its sparse debugger events also rule out a per-kernel OutputDebugString storm. An earlier control that stayed on the startup screen completed zero DLSS frames and is excluded. Normal game launches now omit the attached debugger; crash diagnostics remain available with -CaptureExceptions.

The 2961-frame 4K K profile identifies the translated output tail as its largest kernel: 4.401670 ms mean GPU time, versus about 2.875 ms for all eleven native transformer layers combined. Ported upstream's texture-store recipe to native Windows. It retains accuracy mode, wave32, FP16 subnormals and f16 reference accumulation; native FP8 remains disabled.

Both output variants execute on every checked frame and match the previous RGB output bit-exactly: eight consecutive 512x288 frames per variant, then three consecutive 3840x2160 frames per variant. All RGBA components are finite. Identical synthetic inputs and public NGX flags give these 4K event times (three samples each; host initialization and readback are outside the event):

K output variant	Translated mean, ms	Native stores mean, ms
LDR, regular depth, high-resolution MV	2.907823	1.803233
HDR, inverted depth, low-resolution MV	2.978547	2.088347

A subsequent game run executes 6221 frames with the native HDR output tail, zero backend failures and mean tail GPU time 1.801918 ms. The earlier game profile's 4.401670 ms is from a different scene/scheduling state; use the identical-input table for the controlled comparison, not an inferred FPS multiplier. This profiled game run did not enable the NaN scan.

Reproduce the private texture build and full-frame comparisons:

.\scripts\windows\build-native-texture.ps1 -DlssDll "$PWD\nvngx_dlss.dll"
.\scripts\windows\build-native-texture.ps1 -DlssDll "$PWD\nvngx_dlss.dll" -Kernel hiluma_engine_output_depthreg_mvhi_ldr_max_v2_rel
.\scripts\windows\test-native-texture.ps1 -OutputResolution 3840x2160 -Iterations 3

These objects contain code derived from the locally supplied NVIDIA PTX. They stay in ignored local directories and are excluded from the public package. The installer accepts them with -LocalTextureKernels only after validation, checking the exact DLL identity, object and manifest hashes. M retains its independently validated FP16-equivalent baseline.

The following no-debugger K/4K run completes 9982 frames and 9982 finite GPU scans. Kernel event profiling is disabled; the validated private output tail executes natively. The user reports 43-47 FPS. Mean NGX recording interval is 21.867 ms. No backend failure, CPU image copy or previous-frame output occurs.

Random one-in-64 hook samples then identify contention in the shared routing table used by every intercepted D3D12 command. Per-thread weak lookup caches now bypass that table on repeated calls to a list. Table mutations invalidate cache generations; Reset/address reuse cannot route to a previous suffix, and thread caches do not retain game resources. The per-list and submission locks remain. A GPU regression resets and re-splits the exact same list with changed root constants twice per iteration, checking both prefix and suffix results.

Sampled command-hook access	Shared table	Per-thread weak cache
Busiest recording thread, mean us/call	0.203	0.066
Estimated access + capture across threads, ms/s	977.528	233.606

The latter estimate includes lock waits summed across concurrently running threads, not CPU execution time or serial frame latency. The two game runs have different thread populations (31 versus 29); there is no identical frame trace. Generated forwarding methods have separate driver/capture markers; special-case hooks contribute access samples but lack those body markers.

With the cache, 4770 complete K/4K frames and finite GPU scans pass, with zero backend failures, CPU image copies or previous-frame output. Median NGX recording interval is 19.473 ms. The user reports 49-51 FPS and 53% GPU usage in the previously measured scene. These user readings are separate from the sampled hook estimates and do not establish isolated WMMA utilization. All sixteen CTest gates pass, including 64 same-object Reset/split checks in each physical command-backend test and the software indirect-root regression. Use -ProfileCommandHooks -ProfileStages to collect samples; the sampler is disabled for ordinary play.

Native M FP8 was subsequently compiled and tested on this gfx1201. ISA confirms v_wmma_f32_16x16x16_fp8_fp8, with packed e4m3 weight images; exact FP16 attention/PV remains. All logical outputs and merges in forty recorded launches match the previously NumPy-validated FP16 baseline byte-for-byte. Full DLSS RGB also matches exactly: four 512x288 LDR frames and four 3840x2160 HDR frames, all RGBA finite. This establishes correctness for those fixtures, not a speedup.

The original software operand packing makes the FP8 network slower in the four-frame 4K test: sum of five main-kernel event means (tube weighted six times) is 27.789640 ms versus FP16's 23.550875 ms. A paired in-process replay confirms the regression without relying solely on separate process timings. Both modules and prepared weights remain resident, 512 launches warm the GPU, then sixty-four AB/BA pairs time the same restored input using HIP events. Diagnostic CPU input restoration is outside the timed event and is unrelated to game VRAM interop.

Added opt-in D4R_FP8_HW_PACK: native packed conversion of already quantised operands replaces software exponent/subnormal reconstruction. It preserves all 254 finite OCP e4m3 encodings, including signed zero, in an exhaustive GPU check. All forty recorded launches still match exactly. Four complete 4K HDR frames also match FP16 RGB exactly with finite RGBA. ISA confirms both packed converts and FP8 WMMA. The baseline build's sixteen objects remain byte-identical.

Paired replay median, ms	FP16 (software-packing test)	FP8 software packing	FP16 (hardware-packing test)	FP8 hardware packing
enc1	0.175450	0.217050	0.175100	0.177700
enc2	0.141350	0.164550	0.132550	0.131550
enc3 tube	0.108000	0.112000	0.107600	0.099850
dec2	0.129900	0.146000	0.143250	0.140850
dec1	0.169650	0.194450	0.154250	0.156550

These are two paired tests on the captured 512x288 workload. Medians avoid large isolated scheduling outliers; raw per-pair ratios are retained. They identify a substantial cost in software packing but do not establish an overall FPS gain. The separately run four-frame 4K hardware-packing network mean is 24.603463 ms; it still does not beat the measured FP16 total. Default packaging retains FP16. Hardware counters for bandwidth or WMMA utilization were not measured.

Reproduce the isolated build, encoding/replay checks and paired timings using the existing private validated captures; no source editing is required:

.\scripts\windows\build-native-m-fp8.ps1 -DlssDll "$PWD\nvngx_dlss.dll" -HardwarePacking
.\scripts\windows\test-native-m-fp8.ps1 -ModuleDirectory "$PWD\build\native-m-fp8-hardware-pack" -CaptureDirectory "$PWD\test-results\ngx-m-exact-pv-capture\captures" -BaselineValidation "$PWD\test-results\m-native-exact-pv-temporal-reference\summary.json" -Benchmark

Omit -HardwarePacking to compile the original FP8 experiment. The replay validator stops at the first mismatch, retains GPU stdout/stderr/exit code and hashes every compared logical output. profile_report.py accepts replay logs and reports per-pair GPU time and candidate/control ratios. This experiment is not installed by the game runner or substituted silently for the baseline. The conversion builtin is listed in the official Clang AMDGPU builtin reference.

References: HIP events, HIP occupancy API.