Add RDNA4 (gfx12) support and a 0.1.2 optimization round
RDNA4 (experimental; untested on RDNA4 hardware) - kernels/common/wmma_layout.h picks the WMMA operand/accumulator layout per target; K (pwin_*) and M (swin_*) index through it, so the same sources build for gfx11 and gfx12. D4R_WMMA_LAYOUT=12 on gfx11 is a test shim that runs the gfx12 indexing on gfx11 WMMA. - M can use native e4m3 FP8 WMMA on gfx12 (D4R_NATIVE_FP8=1). Texture tails (enc0_tail, dec0_head) get gfx12 f16 and FP8 paths that match ZLUDA's gfx12 lowering. - ZLUDA patch 0007: gfx12 WMMA helpers, optional native FP8 MMA (D4R_ZLUDA_WMMA_FP8_NATIVE), a layout override for testing, and an architecture argument for d4r_emit. Texture kernels are now compiled offline per target through d4r_emit (D4R_ZLUDA_EMIT) instead of on the local GPU. - New d4r.ini option [Kernels] NativeFp8 (default on), wired through the shim and d4r_config.py. On gfx12 the bridge serves kernels/<arch>-fp8 when it is on. - The bridge picks the KFD GPU with the most SIMDs (so an iGPU is not chosen over a discrete GPU); D4R_GPU_ARCH overrides it. - kernels/build.sh and package_release.sh build gfx1100-gfx1103 and gfx1200/gfx1201 (plus -fp8 variants), and D4R_BUNDLE_TEX takes per-target folders. Performance (RDNA3; harness byte-identical unless noted) - ZLUDA patch 0006: m16n8k8 f16 MMAs on WMMA (E; not bit-exact, 52-55 dB per layer, D4R_ZLUDA_WMMA_K8=0 disables it), per-key weight-image slots for native prep kernels (d4r_prep_key_at, d4r_prep_key_slots), and wave64 builds of MMA-free kernels (D4R_ZLUDA_WAVE64). - M: packed FP8 encoding, conflict-free patch-merge reads, scalar weight loads in enc1, and weight prep once per weight set for the six enc3 tube blocks. - K: CU mode for the six non-wide pwin layers. - Texture kernels: every DLSS flag variant is built natively (24 K output, 8 M post, 4 M downsample, 4 M enc0), not only Townfall's; post and K output run as wave64. - M direct output: rrlite_downsample_kernel writes the shim's output buffer; the shim's default direct-output presets are now 11,13. Tools - dump_runner takes the prep grid from d4r_prep_blocks and refuses texture/surface dumps. - New pwin_check.py, swin_check.py, model_enc3.py and tail_check.hip. Docs: README GPU table and RDNA4 notes, building (patches 0002-0007, d4r_emit), native-kernels (RDNA4 section), performance (new numbers), packaging README. Version 0.1.2.
This commit is contained in:
ayo committed
1 parent
f310d52b69
commit
4334a0c3c4
42 files changed
+3068
-529
No files matched your search
+47
-17
@@ -1,14 +1,19 @@
|
||||
#!/usr/bin/env bash
|
||||
# Builds the native RDNA3 replacements for DLSS kernels into one directory that ZLUDA serves them from
|
||||
# Builds the native replacements for DLSS kernels into one directory that ZLUDA serves them from
|
||||
# (D4R_ZLUDA_NATIVE_DIR, or NativeKernelDirFast in d4r.ini). See docs/native-kernels.md.
|
||||
#
|
||||
# usage: kernels/build.sh [all|k|m|tex] [OUT_DIR] (default: all, kernels/out/native)
|
||||
#
|
||||
# D4R_ROCM_DIR ROCm installation with clang and the HIP device libraries (default /opt/rocm)
|
||||
# D4R_GPU_ARCH target GPU (default gfx1101; the kernels need RDNA3 / gfx11 WMMA)
|
||||
# D4R_GPU_ARCH target GPU (default gfx1101): an RDNA3 (gfx110x) or RDNA4 (gfx120x) target; the kernels
|
||||
# need wave32 WMMA, and kernels/common/wmma_layout.h picks the register layout per target
|
||||
# D4R_NATIVE_FP8 1 (RDNA4 only): the variant for ZLUDA's native FP8 WMMA (d4r.ini [Kernels] NativeFp8):
|
||||
# M's layers on FP8 WMMA with NVIDIA's e4m3 activations, texture tails matching ZLUDA's
|
||||
# D4R_ZLUDA_WMMA_FP8_NATIVE lowering. K is the same in both variants.
|
||||
# Texture kernels (tex) are NVIDIA's PTX with parts replaced, so they additionally need:
|
||||
# D4R_DLSS_DLL nvngx_dlss.dll 310.7 (its PTX is extracted into kernels/extracted/, never committed)
|
||||
# D4R_ZLUDA_BUILD a ZLUDA build with patches/zluda applied (directory holding libcuda.so)
|
||||
# D4R_ZLUDA_EMIT ZLUDA's d4r_emit (patches/zluda applied; `cargo build --release -p ptx --example d4r_emit`),
|
||||
# which compiles PTX offline for D4R_GPU_ARCH (no GPU of that kind needed)
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
@@ -16,17 +21,26 @@ WHAT="${1:-all}"
|
||||
OUT="$(realpath -m "${2:-$HERE/out/native}")"
|
||||
ROCM="${D4R_ROCM_DIR:-/opt/rocm}"
|
||||
ARCH="${D4R_GPU_ARCH:-gfx1101}"
|
||||
FP8="${D4R_NATIVE_FP8:-0}"
|
||||
case "$ARCH" in
|
||||
gfx11[0-9][0-9]) [[ "$FP8" == 1 ]] && { echo "D4R_NATIVE_FP8 needs an RDNA4 (gfx12) target" >&2; exit 2; } ;;
|
||||
gfx12[0-9][0-9]) ;;
|
||||
*) echo "unsupported D4R_GPU_ARCH $ARCH (RDNA3 gfx110x or RDNA4 gfx120x)" >&2; exit 2 ;;
|
||||
esac
|
||||
CLANG="$ROCM/lib/llvm/bin/clang++"
|
||||
[[ -x "$CLANG" ]] || { echo "clang++ not found in $ROCM/lib/llvm/bin (set D4R_ROCM_DIR)" >&2; exit 2; }
|
||||
mkdir -p "$OUT"
|
||||
|
||||
# HIP source -> raw code object (the kernel name inside matches the DLSS kernel it replaces)
|
||||
build_hip() {
|
||||
local src="$1" name
|
||||
local src="$1" extra="${2:-}" name
|
||||
name="$(basename "$src" .hip)"
|
||||
local tmp
|
||||
local tmp flags
|
||||
tmp="$(mktemp -d)"
|
||||
(cd "$tmp" && "$CLANG" -x hip --offload-arch="$ARCH" --offload-device-only -O3 -I"$ROCM/include" \
|
||||
# per-kernel compiler flags from a "// d4r-build-flags: ..." line in the source
|
||||
flags="$(sed -n 's|^// d4r-build-flags: *||p' "$src")"
|
||||
# shellcheck disable=SC2086
|
||||
(cd "$tmp" && "$CLANG" -x hip --offload-arch="$ARCH" --offload-device-only -O3 $flags $extra -I"$ROCM/include" \
|
||||
--rocm-path="$ROCM" --rocm-device-lib-path="$ROCM/amdgcn/bitcode" -I"$(dirname "$src")" \
|
||||
-o "$OUT/$name.hsaco" "$src" -save-temps=cwd --no-gpu-bundle-output)
|
||||
printf '%-52s %s\n' "$name" "$(grep -hE '^; (NumVgprs|ScratchSize|Occupancy)' "$tmp"/*.s | tail -3 | paste -sd' ')"
|
||||
@@ -39,23 +53,39 @@ if [[ "$WHAT" == all || "$WHAT" == k ]]; then
|
||||
fi
|
||||
if [[ "$WHAT" == all || "$WHAT" == m ]]; then
|
||||
echo "== DLSS 4.5 (preset M) Swin layers"
|
||||
for src in "$HERE"/m/rrlite_*.hip; do build_hip "$src"; done
|
||||
for src in "$HERE"/m/rrlite_*.hip; do build_hip "$src" "$([[ "$FP8" == 1 ]] && echo -DD4R_FP8_WMMA)"; done
|
||||
fi
|
||||
if [[ "$WHAT" == all || "$WHAT" == tex ]]; then
|
||||
echo "== texture kernels (NVIDIA PTX with native parts)"
|
||||
: "${D4R_DLSS_DLL:?set D4R_DLSS_DLL to nvngx_dlss.dll (310.7)}"
|
||||
: "${D4R_ZLUDA_BUILD:?set D4R_ZLUDA_BUILD to a ZLUDA build with patches/zluda}"
|
||||
: "${D4R_ZLUDA_EMIT:?set D4R_ZLUDA_EMIT to d4r_emit from a ZLUDA build with patches/zluda}"
|
||||
PTX_DIR="$HERE/extracted/ptx"
|
||||
python3 "$HERE/tools/extract_dlss_ptx.py" "$D4R_DLSS_DLL" "$PTX_DIR"
|
||||
PTXLOAD="$HERE/out/tools/ptxload"
|
||||
mkdir -p "$(dirname "$PTXLOAD")"
|
||||
cc -O2 -o "$PTXLOAD" "$HERE/tools/ptxload.c" -ldl
|
||||
# kernel : source of its native part
|
||||
for pair in hiluma_engine_output_depthinv_mvhi_hdr_max_v2_rel:sust_only rrlite_post_3_2_mvhi_hdr_folded:sust_only \
|
||||
rrlite_downsample_kernel_static_hdr:sust_only rrlite_enc0_4x4_mvhi_hdr_folded:enc0_tail \
|
||||
rrlite_dec0_4x4_folded:dec0_head; do
|
||||
D4R_ROCM_DIR="$ROCM" D4R_GPU_ARCH="$ARCH" D4R_DLSS_PTX_DIR="$PTX_DIR" D4R_PTXLOAD="$PTXLOAD" \
|
||||
"$HERE/tex/build_tex.sh" "${pair%%:*}" "${pair#*:}" "$OUT"
|
||||
# kernel : source of its native part [: w64 = compiled as wave64 (two CUDA warps per wave; RDNA3 issues
|
||||
# FP32 work for all 64 lanes at once). Only for kernels without MMAs; measured faster, bit-identical:
|
||||
# post 0.89 -> 0.77 ms, hiluma output 0.905 -> 0.887 ms (downsample was slower as wave64).]
|
||||
# Every flag combination DLSS selects (motion vectors hi/lo, HDR/LDR, depth inverted/regular, ...) gets
|
||||
# the same treatment, so games other than the one measured run the native versions too.
|
||||
specs=(rrlite_dec0_4x4_folded:dec0_head)
|
||||
for mv in mvhi mvlo; do
|
||||
for range in hdr ldr; do
|
||||
specs+=("rrlite_enc0_4x4_${mv}_${range}_folded:enc0_tail")
|
||||
for v in 3_1 3_2; do specs+=("rrlite_post_${v}_${mv}_${range}_folded:sust_only:w64"); done
|
||||
for depth in depthinv depthreg; do
|
||||
for kind in "" _max; do
|
||||
[[ -z "$kind" ]] && specs+=("hiluma_engine_output_${depth}_${mv}_${range}_v1_rel:sust_only:w64")
|
||||
specs+=("hiluma_engine_output_${depth}_${mv}_${range}${kind}_v2_rel:sust_only:w64")
|
||||
done
|
||||
done
|
||||
done
|
||||
done
|
||||
for mode in static dynamic; do
|
||||
for range in hdr ldr; do specs+=("rrlite_downsample_kernel_${mode}_${range}:sust_only"); done
|
||||
done
|
||||
for spec in "${specs[@]}"; do
|
||||
IFS=: read -r kernel src mode <<< "$spec"
|
||||
D4R_ZLUDA_WAVE64="$([[ "$mode" == w64 ]] && echo 1 || echo 0)" D4R_ROCM_DIR="$ROCM" D4R_GPU_ARCH="$ARCH" \
|
||||
D4R_TEX_FP8="$FP8" D4R_DLSS_PTX_DIR="$PTX_DIR" "$HERE/tex/build_tex.sh" "$kernel" "$src" "$OUT"
|
||||
done
|
||||
fi
|
||||
echo "native kernels in $OUT"
|
||||
Reference in new issue
Block a user