woutermans
1038825225
make_ptx.py turns the 32 cvt.rn.f16x2.e4m3x2 of rrlite_dec0_4x4_folded into calls of d4r_dec_e4m3x2 (tex_common.h) when D4R_TEX_FP8=1: v_cvt_pk_f32_fp8 and two conversions to f16 instead of ZLUDA's integer lowering (about 25 instructions each; the kernel goes from 4058 to 3418 instructions). The conversion equals ZLUDA's e4m3_to_f16_bits for every code except the NaN codes 0x7f / 0xff, checked over all 65536 code pairs on an RX 9070 XT, and the head never stages those codes (it clamps to +-448 before encoding). Harness frames are byte-identical in the accuracy and the fast set, over a 16-frame moving sequence in the accuracy set. dec0 at 2227x1253 render: 0.43 -> 0.38 ms (accuracy), 0.36 -> 0.30 ms (fast). -DD4R_TEX_SOFT_E4M3 gives ZLUDA's lowering; builds without D4R_TEX_FP8 keep the PTX instruction.
39 lines
2.6 KiB
Bash
Executable File
39 lines
2.6 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# build_tex.sh KERNEL SRC OUT_DIR: one texture kernel. NVIDIA's PTX for KERNEL, edited by make_ptx.py so
|
|
# part of it calls functions from SRC.hip, is compiled by ZLUDA with SRC's bitcode linked in
|
|
# (D4R_ZLUDA_EXTRA_BC) and the resulting code object is saved as OUT_DIR/KERNEL.hsaco.
|
|
# Called by kernels/build.sh, which sets D4R_ROCM_DIR, D4R_GPU_ARCH, D4R_DLSS_PTX_DIR, D4R_ZLUDA_EMIT,
|
|
# D4R_ZLUDA_WAVE64 (1 = wave64 code object; accuracy builds only with D4R_TEX_ACCURACY_WAVE64=1) and
|
|
# D4R_TEX_FP8 (1 = the gfx12 native-FP8 variant).
|
|
# ZLUDA compiles offline for D4R_GPU_ARCH with its d4r_emit tool, so any target builds on any machine.
|
|
set -euo pipefail
|
|
K=$1 SRC=$2 OUT=$3
|
|
D="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
R="${D4R_ROCM_DIR:-/opt/rocm}"
|
|
ARCH="${D4R_GPU_ARCH:-gfx1101}"
|
|
FP8="${D4R_TEX_FP8:-0}"
|
|
ACCURACY="${D4R_PREFER_ACCURACY:-0}"
|
|
LLVM="$R/lib/llvm/bin"
|
|
B="$(mktemp -d)"
|
|
trap 'rm -rf "$B"' EXIT
|
|
# D4R_TEX_CFLAGS: extra flags for SRC.hip (tests, e.g. -DD4R_WMMA_LAYOUT=12 for the gfx12-layout shim on gfx11)
|
|
read -r -a FLAGS <<< "${D4R_TEX_CFLAGS:-}"
|
|
[[ "$FP8" == 1 ]] && FLAGS+=(-DD4R_TEX_FP8)
|
|
[[ "$ACCURACY" == 1 ]] && FLAGS+=(-DD4R_ACCURACY)
|
|
# SRC.hip -> bitcode without target attributes, so ZLUDA's own target settings apply after linking
|
|
"$LLVM/clang++" -x hip -std=c++20 -nogpuinc -nogpulib -O3 -mno-wavefrontsize64 --offload-device-only \
|
|
--offload-arch="$ARCH" -fgpu-rdc -emit-llvm -c -Xclang -fdenormal-fp-math=dynamic -DD4R_KERNEL_NAME="$K" \
|
|
"${FLAGS[@]}" -o "$B/raw.bc" "$D/$SRC.hip"
|
|
"$LLVM/llvm-dis" "$B/raw.bc" -o "$B/raw.ll"
|
|
sed -E -e '/@llvm.used/d' -e '/wchar_size/d' -e '/llvm.module.flags/d' -e '/__hip_cuid/d' -e 's/optnone//g' \
|
|
-e "s/\"target-cpu\"=\"$ARCH\"//g" -e 's/"target-features"="[^"]+"//g' "$B/raw.ll" | "$LLVM/llvm-as" -o "$B/extra.bc" -
|
|
D4R_PREFER_ACCURACY="$ACCURACY" D4R_TEX_FP8="$FP8" python3 "$D/make_ptx.py" "$K" "$B/$K.ptx" >/dev/null
|
|
# the code-generation switches of the runtime (d4r.ini [Kernels]), which ZLUDA's module cache keys on
|
|
env LD_LIBRARY_PATH="$R/lib" D4R_ZLUDA_EXTRA_BC="$B/extra.bc" D4R_ZLUDA_WMMA=1 D4R_ZLUDA_WMMA_FP8=1 \
|
|
D4R_ZLUDA_WMMA_FP8_NATIVE="$FP8" D4R_ZLUDA_IGNORE_DENORMAL="$((1 - ACCURACY))" \
|
|
D4R_ZLUDA_WMMA_F32ACC=0 D4R_ZLUDA_FAST_MATH=0 D4R_ZLUDA_IMPLICIT_MAX_BLOCK=256 \
|
|
D4R_ZLUDA_WAVE64="$([[ "$ACCURACY" == 1 && "${D4R_TEX_ACCURACY_WAVE64:-0}" != 1 ]] && echo 0 || echo "${D4R_ZLUDA_WAVE64:-0}")" \
|
|
"$D4R_ZLUDA_EMIT" "$B/$K.ptx" "$B/dump" "$ARCH" >/dev/null
|
|
cp "$B/dump/module.hsaco" "$OUT/$K.hsaco"
|
|
printf '%-52s %s\n' "$K" "$(grep -E '^\s+\.(vgpr_count|private_segment_fixed_size|wavefront_size):' "$B/dump/asm.s" | tr -s ' ' | paste -sd' ')"
|