Add exact input-kernel experiments and preserve translated controls
This commit is contained in:
xdfnx-dev committed
1 parent
4f008b3ed1
commit
8f7058d2ba
5 files changed
+102
-16
No files matched your search
@@ -601,6 +601,59 @@ the game's non-aligned input dimensions without inventing a larger output.
|
||||
An initial attempt using 4518x2542 output is rejected by the harness's existing
|
||||
range guard before GPU initialization; it is not a GPU or enc0 failure.
|
||||
|
||||
### Rejected K input-rounding overrides
|
||||
|
||||
The remaining translated K input is about 0.34-0.40 ms in the earlier
|
||||
game-sized standalone profile. The private builder now exposes the existing
|
||||
surface-store/rounding recipe for both tested input variants. These inputs
|
||||
have **zero surface stores**: the actual change replaces eight round-half-away
|
||||
idioms with the existing recipe, avoiding constrained arithmetic lowering.
|
||||
It is not a native surface-store saving. Build metadata records the recipe's
|
||||
counts and refuses a no-op override.
|
||||
|
||||
CU and WGP input objects are built from the same clean `fb0adc8` ZLUDA source.
|
||||
Both retain accuracy mode and wave32; native FP8 remains disabled. The full
|
||||
pipeline control and candidate share all thirteen other code objects,
|
||||
including the same private WGP output tails. Only the two input overrides
|
||||
are added. Four frames per variant at 512x288, then eight per variant at
|
||||
2259x1271 -> 3840x2160 in CU mode, then eight in WGP mode with candidate-first
|
||||
order complete **80 finite control/candidate frames**. All forty candidate
|
||||
RGB images match exactly. Every expected input launch and valid serializing
|
||||
timing is checked; raw logs and module SHA256 values are retained.
|
||||
|
||||
| Input median, ms | Translated, CU test | Native CU | Translated, WGP test | Native WGP |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| LDR / regular depth / high-resolution MV | 0.350050 | 0.359425 | 0.351000 | 0.358800 |
|
||||
| HDR / inverted depth / low-resolution MV | 0.512300 | 0.556350 | 0.412650 | 0.552100 |
|
||||
|
||||
These are separate-process diagnostic measurements with eight samples each;
|
||||
other unchanged kernels also vary between processes. Neither input candidate
|
||||
shows a saving. Both are rejected for game installation and default packaging;
|
||||
the validated game configuration keeps translated input. Results:
|
||||
`test-results/k-input-native-stores-small` (initial filename; no stores were
|
||||
replaced), `k-input-rounding-cu-game-input`,
|
||||
`k-input-rounding-wgp-game-input-reverse` and
|
||||
`k-input-rounding-experiment-summary.json`.
|
||||
|
||||
The comparison script's `-KernelStage input` selects the input hit/timing gate.
|
||||
`-CommonTextureRoot` holds output objects identical in both processes; staged
|
||||
object hashes are recorded. The ordinary output gate now accepts a translated
|
||||
control when its object is absent, while still requiring a native candidate.
|
||||
Rebuilding the two original WGP output objects preserves their SHA256 values;
|
||||
the default translated-output comparison also passes four finite frames and
|
||||
two exact candidate RGB images
|
||||
(`k-output-recipe-translated-control-regression`). Input objects remain private
|
||||
and are excluded by the game's existing output-only installation allowlist.
|
||||
|
||||
Reproduce the WGP experiment with locally built, validated output tails:
|
||||
|
||||
```powershell
|
||||
foreach ($kernel in @('hiluma_engine_input_depthinv_mvlo_hdr_v2_rel', 'hiluma_engine_input_depthreg_mvhi_ldr_v2_rel')) {
|
||||
.\scripts\windows\build-native-texture.ps1 -DlssDll "$PWD\nvngx_dlss.dll" -ZludaRoot "$PWD\dist\zluda-windows-wgp-experiment" -ShaderMode wgp -Kernel $kernel -OutputDirectory "$PWD\build\private-input-textures-wgp-gfx1201"
|
||||
}
|
||||
.\scripts\windows\test-native-texture.ps1 -KernelStage input -TextureRoot "$PWD\build\private-input-textures-wgp-gfx1201" -CommonTextureRoot "$PWD\build\private-textures-wgp-gfx1201" -ZludaRoot "$PWD\dist\zluda-windows-deferred-profile" -InputResolution 2259x1271 -OutputResolution 3840x2160 -Iterations 8 -CandidateFirst -OutputDirectory "$PWD\test-results\k-input-rounding-wgp-retest"
|
||||
```
|
||||
|
||||
Further game coverage (HIP kernel profiling disabled, GPU output checks enabled):
|
||||
M at 1920x1080 completes 4833 frames, all finite. K at 3840x2160 completes
|
||||
4685 frames, all finite. Neither has a backend failure, CPU image copy or an
|
||||
|
||||
@@ -76,6 +76,15 @@ probe/game children so disabled diagnostic flags stay disabled; GPU comparisons
|
||||
now require actual valid timing samples. Commands, rejected experiments and
|
||||
the initial diagnostic/cache failures are recorded in `windows-performance.md`.
|
||||
|
||||
A subsequent input-rounding CU/WGP experiment completes eighty finite frames
|
||||
with all forty candidate RGB images exact, holding the other thirteen objects
|
||||
identical. Neither input candidate improves standalone timing; the game keeps
|
||||
translated input. The private recipe/test tools retain these reproducible
|
||||
experiments, while the installer still accepts only the two validated output
|
||||
variants. Rebuilt output objects retain their SHA256 values, and the ordinary
|
||||
translated-output control comparison passes four further finite frames. See
|
||||
[windows-performance.md](windows-performance.md#rejected-k-input-rounding-overrides).
|
||||
|
||||
The next diagnostics add `-ProfileCudaApi` (ZLUDA patch 0015): aggregate host
|
||||
CUDA API durations without HIP-event waits. Boundary stages now separate
|
||||
prepare, copy-list setup, input/output submission and output-copy drain.
|
||||
|
||||
@@ -209,8 +209,9 @@ def replace_rz_round(lines):
|
||||
|
||||
|
||||
ENC0_REF = "rrlite_enc0_4x4_mvhi_hdr_folded"
|
||||
# the other flag combinations of the kernels whose only native part is the surface-store rewrite
|
||||
# Surface-store and round-idiom recipes for the other flag combinations.
|
||||
SUST_ONLY_RE = re.compile(r"^(hiluma_engine_output_depth(inv|reg)_mv(hi|lo)_(hdr|ldr)(_max)?_v[12]_rel|"
|
||||
r"hiluma_engine_input_depth(inv|reg)_mv(hi|lo)_(hdr|ldr)_v2_rel|"
|
||||
r"rrlite_post_3_[12]_mv(hi|lo)_(hdr|ldr)(_folded)?|"
|
||||
r"rrlite_enc0_4x4_mv(hi|lo)_(hdr|ldr)|rrlite_dec0_4x4|"
|
||||
r"rrlite_downsample_kernel_(static|dynamic)_(hdr|ldr))$")
|
||||
@@ -277,7 +278,7 @@ def kernel_spec(name):
|
||||
if ENC0_RE.match(name):
|
||||
return enc0_variant(name)
|
||||
if SUST_ONLY_RE.match(name):
|
||||
return dict(file="", sust=True, extern="", rz_round=name.startswith("hiluma_engine_output"))
|
||||
return dict(file="", sust=True, extern="", rz_round=name.startswith("hiluma_engine_"))
|
||||
sys.exit(f"make_ptx: no recipe for {name}")
|
||||
|
||||
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
[CmdletBinding()]
|
||||
param([string]$TextureRoot, [string]$ControlTextureRoot, [string]$ZludaRoot,
|
||||
param([string]$TextureRoot, [string]$ControlTextureRoot, [string]$CommonTextureRoot, [string]$ZludaRoot,
|
||||
[string]$OutputDirectory, [int]$Iterations=8, [string]$OutputResolution='512x288', [string]$InputResolution,
|
||||
[switch]$CandidateFirst)
|
||||
[switch]$CandidateFirst,
|
||||
[ValidateSet('output','input')][string]$KernelStage='output')
|
||||
$ErrorActionPreference='Stop'
|
||||
$repo=[IO.Path]::GetFullPath((Join-Path $PSScriptRoot '../..'))
|
||||
if (!$TextureRoot) { $TextureRoot=Join-Path $repo 'build/private-textures-gfx1201' }
|
||||
@@ -11,16 +12,16 @@ if ($OutputResolution -notmatch '^([0-9]+)x([0-9]+)$') { throw 'OutputResolution
|
||||
$outputWidth=[int]$Matches[1]; $outputHeight=[int]$Matches[2]
|
||||
$combined=Join-Path $OutputDirectory 'private-native'
|
||||
New-Item -ItemType Directory -Force $combined | Out-Null
|
||||
foreach ($directory in @((Join-Path $repo 'build/native-k-gfx1201'),$TextureRoot)) {
|
||||
foreach ($directory in @((Join-Path $repo 'build/native-k-gfx1201'),$CommonTextureRoot,$TextureRoot) | Where-Object { $_ }) {
|
||||
Get-ChildItem -LiteralPath $directory -Filter '*.hsaco' | Copy-Item -Destination $combined -Force
|
||||
}
|
||||
& python (Join-Path $repo 'kernels/tools/kernel_manifest.py') $combined (Join-Path $repo 'nvngx_dlss.dll')
|
||||
if ($LASTEXITCODE) { throw 'Combined private manifest failed.' }
|
||||
$controlNative=Join-Path $repo 'build/native-k-gfx1201'
|
||||
if ($ControlTextureRoot) {
|
||||
if ($ControlTextureRoot -or $CommonTextureRoot) {
|
||||
$controlNative=Join-Path $OutputDirectory 'control-private-native'
|
||||
New-Item -ItemType Directory -Force $controlNative | Out-Null
|
||||
foreach ($directory in @((Join-Path $repo 'build/native-k-gfx1201'),$ControlTextureRoot)) {
|
||||
foreach ($directory in @((Join-Path $repo 'build/native-k-gfx1201'),$CommonTextureRoot,$ControlTextureRoot) | Where-Object { $_ }) {
|
||||
Get-ChildItem -LiteralPath $directory -Filter '*.hsaco' | Copy-Item -Destination $controlNative -Force
|
||||
}
|
||||
& python (Join-Path $repo 'kernels/tools/kernel_manifest.py') $controlNative (Join-Path $repo 'nvngx_dlss.dll')
|
||||
@@ -32,13 +33,24 @@ $settings=@{D4R_VALIDATE_OUTPUT='1'; D4R_PROFILE_STAGES='1'; D4R_ZLUDA_PROFILE='
|
||||
ZLUDA_CACHE_DIR=(Join-Path $repo 'build/zluda-cache-windows'); PYTHONPATH=(Join-Path $repo '.tools/python/vendor');
|
||||
D4R_ZLUDA_NATIVE_DIR=$null}
|
||||
$old=@{}
|
||||
$stagedObjects=@{}
|
||||
foreach ($stage in @(@{name='control'; root=$controlNative},@{name='candidate'; root=$combined})) {
|
||||
$stagedObjects[$stage.name]=@{}
|
||||
foreach ($object in Get-ChildItem -LiteralPath $stage.root -Filter '*.hsaco') {
|
||||
$stagedObjects[$stage.name][$object.Name]=(Get-FileHash -LiteralPath $object.FullName -Algorithm SHA256).Hash.ToLowerInvariant()
|
||||
}
|
||||
}
|
||||
try {
|
||||
foreach ($key in $settings.Keys) { $old[$key]=[Environment]::GetEnvironmentVariable($key,'Process'); [Environment]::SetEnvironmentVariable($key,$settings[$key],'Process') }
|
||||
foreach ($flags in @(0,11)) {
|
||||
$kernel=if ($flags -eq 11) { 'hiluma_engine_output_depthinv_mvlo_hdr_max_v2_rel' } else { 'hiluma_engine_output_depthreg_mvhi_ldr_max_v2_rel' }
|
||||
$variant=if ($flags -eq 11) { 'depthinv_mvlo_hdr' } else { 'depthreg_mvhi_ldr' }
|
||||
$suffix=if ($KernelStage -eq 'output') { '_max_v2_rel' } else { '_v2_rel' }
|
||||
$kernel="hiluma_engine_${KernelStage}_${variant}${suffix}"
|
||||
$order=if ($CandidateFirst) { @('candidate','control') } else { @('control','candidate') }
|
||||
foreach ($mode in $order) {
|
||||
$env:D4R_ZLUDA_NATIVE_DIR=if ($mode -eq 'control') { $controlNative } else { $combined }
|
||||
$expectedBackend=if (Test-Path -LiteralPath (Join-Path $env:D4R_ZLUDA_NATIVE_DIR ($kernel + '.hsaco'))) { 'native' } else { 'translated' }
|
||||
if ($mode -eq 'candidate' -and $expectedBackend -ne 'native') { throw "Missing candidate object: $kernel" }
|
||||
$arguments=@{RuntimeProfile='therock'; ZludaRoot=$ZludaRoot;
|
||||
PackageRoot=(Join-Path $repo 'dist/windows-rdna4-command-list'); NgxCore=(Join-Path $repo '_nvngx.dll');
|
||||
DlssDll=(Join-Path $repo 'nvngx_dlss.dll'); NgxMode='d3d12'; NgxAbi='project-legacy'; Preset=11;
|
||||
@@ -52,9 +64,9 @@ try {
|
||||
& python (Join-Path $repo 'tools/windows/profile_report.py') $arguments.OutputDirectory --output (Join-Path $arguments.OutputDirectory 'profile.json')
|
||||
if ($LASTEXITCODE) { throw 'Texture profile report failed.' }
|
||||
$profile=Get-Content -LiteralPath (Join-Path $arguments.OutputDirectory 'profile.json') -Raw | ConvertFrom-Json
|
||||
$samples=@($profile.kernels | Where-Object { $_.kernel -eq $kernel -and $_.backend -eq 'native' -and $_.phase -eq 'main' -and $_.serializing })
|
||||
$samples=@($profile.kernels | Where-Object { $_.kernel -eq $kernel -and $_.backend -eq $expectedBackend -and $_.phase -eq 'main' -and $_.serializing })
|
||||
if ($samples.Count -ne 1 -or $samples[0].gpuMs.samples -ne $Iterations) {
|
||||
throw "Missing valid serializing output-kernel timings for $mode flags=$flags; retain this run as correctness evidence only."
|
||||
throw "Missing valid serializing $KernelStage-kernel timings ($expectedBackend) for $mode flags=$flags; retain this run as correctness evidence only."
|
||||
}
|
||||
}
|
||||
$control=Join-Path $OutputDirectory "control-$flags"; $candidate=Join-Path $OutputDirectory "candidate-$flags"
|
||||
@@ -66,12 +78,13 @@ try {
|
||||
}
|
||||
@{passed=$true; architecture='gfx1201'; framesPerVariant=$Iterations; outputResolution=$OutputResolution; createFlags=@(0,11); strictRgb=$true;
|
||||
inputResolution=$InputResolution;
|
||||
serializingProfile=$true; candidateFirst=[bool]$CandidateFirst;
|
||||
controlTextureRoot=$ControlTextureRoot;
|
||||
serializingProfile=$true; candidateFirst=[bool]$CandidateFirst; kernelStage=$KernelStage;
|
||||
controlTextureRoot=$ControlTextureRoot; commonTextureRoot=$CommonTextureRoot;
|
||||
stagedObjects=$stagedObjects;
|
||||
manifestSha256=(Get-FileHash -LiteralPath (Join-Path $TextureRoot 'd4r-kernels.txt') -Algorithm SHA256).Hash.ToLowerInvariant();
|
||||
privateNvidiaDerivedCode=$true; source=(Get-Content -LiteralPath (Join-Path $TextureRoot 'build-info.json') -Raw | ConvertFrom-Json)} |
|
||||
ConvertTo-Json -Depth 12 | Set-Content -LiteralPath (Join-Path $TextureRoot 'validation.json') -Encoding UTF8
|
||||
Write-Host "PASS: both private K output variants executed and matched every RGB component. $OutputDirectory"
|
||||
Write-Host "PASS: both private K $KernelStage variants executed and matched every RGB component. $OutputDirectory"
|
||||
} finally {
|
||||
foreach ($key in $old.Keys) { [Environment]::SetEnvironmentVariable($key,$old[$key],'Process') }
|
||||
}
|
||||
@@ -4,6 +4,7 @@ The resulting PTX/code object contains NVIDIA code and must remain local.
|
||||
No Wine, shell script, GPU, CUDA SDK or system installation is required.
|
||||
"""
|
||||
import argparse
|
||||
import ast
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
@@ -23,8 +24,9 @@ def main():
|
||||
parser.add_argument('--output-directory', type=pathlib.Path, required=True)
|
||||
parser.add_argument('--shader-mode', choices=['cu', 'wgp'], default='cu')
|
||||
args = parser.parse_args()
|
||||
if not re.fullmatch(r'hiluma_engine_output_depth(?:inv|reg)_mv(?:hi|lo)_(?:hdr|ldr)(?:_max)?_v2_rel', args.kernel):
|
||||
parser.error('this initial validated build recipe supports K v2 output variants only')
|
||||
if not re.fullmatch(r'hiluma_engine_(?:output_depth(?:inv|reg)_mv(?:hi|lo)_(?:hdr|ldr)(?:_max)?|'
|
||||
r'input_depth(?:inv|reg)_mv(?:hi|lo)_(?:hdr|ldr))_v2_rel', args.kernel):
|
||||
parser.error('this private experimental build recipe supports K v2 input/output variants only')
|
||||
repo = pathlib.Path(__file__).resolve().parents[2]
|
||||
output = args.output_directory.resolve()
|
||||
output.mkdir(parents=True, exist_ok=True)
|
||||
@@ -59,6 +61,13 @@ def main():
|
||||
env['D4R_DLSS_PTX_DIR'] = str(ptx)
|
||||
edited = work / (args.kernel + '.ptx')
|
||||
run('make-ptx', [sys.executable, repo / 'kernels/tex/make_ptx.py', args.kernel, edited])
|
||||
rewrite_match = re.search(r'surface stores replaced: (\{[^\n]+\})',
|
||||
(work / 'make-ptx.log').read_text(errors='replace'))
|
||||
if not rewrite_match:
|
||||
raise RuntimeError('PTX recipe did not report its changes')
|
||||
rewrite_counts = ast.literal_eval(rewrite_match[1])
|
||||
if not any(rewrite_counts.values()):
|
||||
raise RuntimeError('PTX recipe made no changes; refuse a no-op override')
|
||||
raw, ir, extra = work / 'raw.bc', work / 'raw.ll', work / 'extra.bc'
|
||||
run('compile-helper', [tools['clang++'], '-x', 'hip', '-std=c++20', '-nogpuinc', '-nogpulib', '-O3',
|
||||
'-mno-wavefrontsize64', '--offload-device-only', '--offload-arch=gfx1201',
|
||||
@@ -87,13 +96,14 @@ def main():
|
||||
shutil.copyfile(work / 'emitted/module.hsaco', output / (args.kernel + '.hsaco'))
|
||||
run('manifest', [sys.executable, repo / 'kernels/tools/kernel_manifest.py', output, args.dlss_dll])
|
||||
metadata = dict(architecture='gfx1201', accuracy=True, nativeFP8=False, shaderMode=args.shader_mode,
|
||||
lastBuiltKernel=args.kernel, lastRecipeRewrites=rewrite_counts,
|
||||
privateNvidiaDerivedCode=True, validation='not yet validated; do not install before comparison',
|
||||
dlssSha256=hashlib.sha256(args.dlss_dll.read_bytes()).hexdigest(),
|
||||
zludaBuild=json.loads((zluda / 'build-info.json').read_text(encoding='utf-8-sig')),
|
||||
compiler=subprocess.check_output([str(tools['clang++']), '--version'], env=env).decode(errors='replace'),
|
||||
objects={p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in output.glob('*.hsaco')})
|
||||
(output / 'build-info.json').write_text(json.dumps(metadata, indent=2) + '\n')
|
||||
print(f'Built PRIVATE gfx1201 output override: {output}; validation required before use')
|
||||
print(f'Built PRIVATE gfx1201 texture override: {output}; validation required before use')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
||||
Reference in new issue
Block a user