Measure Windows submission costs and validate faster private K output stores
This commit is contained in:
1 parent
d556747bad
commit
73ee2e5ecc
14 files changed
+453
-24
No files matched your search
+20
-5
@@ -3,8 +3,8 @@
|
||||
Validated hardware: Windows 11 x64 / Radeon RX 9070 XT / gfx1201. Native K and
|
||||
M transformer kernels pass standalone D3D12 + OptiScaler checks. Silent Hill 2
|
||||
K now renders real menu frames and M renders a saved gameplay level with
|
||||
GPU NaN/Inf checks. Resolution coverage and performance are still being
|
||||
validated. This is a local
|
||||
GPU NaN/Inf checks. K also passes 3840x2160 output and M passes 1920x1080.
|
||||
Further performance work is in progress. This is a local
|
||||
development package; no NVIDIA proprietary DLL is included.
|
||||
|
||||
Build the shim/diagnostics, corrected ZLUDA, K/M native objects and patched
|
||||
@@ -29,8 +29,10 @@ Use `-Preset 13` for M. The validated manifest requires DLSS 310.9.1 with the
|
||||
recorded SHA256; other feature DLLs require regenerating and validating the
|
||||
native manifest. Select an upscaler supported by OptiScaler in the game.
|
||||
The script installs `dxgi.dll`, an explicit DLSS configuration and a private
|
||||
`d4r` runtime directory, and captures stdout/stderr, an external debugger log,
|
||||
crash dump, loaded DLL paths, driver information and OptiScaler logs. A local
|
||||
`d4r` runtime directory, and captures stdout/stderr, exit code, loaded DLL paths,
|
||||
driver information and OptiScaler logs. Ordinary runs use a process job without
|
||||
an attached debugger. Add `-CaptureExceptions` to capture exception events and
|
||||
minidumps when reproducing a crash. A local
|
||||
ZIP bundle is printed at exit. `-RunSeconds 120` bounds a diagnostic run;
|
||||
the normal default allows you to play until you close the game.
|
||||
|
||||
@@ -46,7 +48,11 @@ the GPU before returning it to D3D12. Only two diagnostic counters (8 bytes)
|
||||
are read by the CPU; image data remains in VRAM. Validation failures are
|
||||
reported explicitly. Disable this diagnostic when measuring production speed.
|
||||
|
||||
`-ProfileStages` records CPU completion time for seven interop/NGX stages.
|
||||
`-ProfileStages` records CPU completion time for seven interop/NGX stages,
|
||||
command-list split/replay, submitted state publication and suffix completion.
|
||||
The recording interval measures NGX calls on the recording thread, not Present.
|
||||
`-ProfileKernels` enables synchronizing HIP-event kernel profiling and changes
|
||||
frame scheduling; use it separately from a production FPS measurement.
|
||||
The runner drains stdout/stderr concurrently as raw bytes; the earlier
|
||||
line-oriented PowerShell collector imposed large per-frame delays. The optional
|
||||
ZLUDA kernel profiler is for diagnostics only. Measurements and reproduction
|
||||
@@ -58,6 +64,15 @@ kernel launches remain logged. `-VerboseRuntime` enables all API and frontend
|
||||
trace messages when investigating a failure. Numerical relaxation flags are
|
||||
cleared in the child process; FP16 rounding and subnormals stay enabled.
|
||||
|
||||
The optional K output-store optimization is built locally from your NVIDIA DLL
|
||||
with `build-native-texture.ps1` and checked by `test-native-texture.ps1`;
|
||||
commands and measurements are in [windows-performance.md](windows-performance.md).
|
||||
Add `-LocalTextureKernels "C:\Users\Administrator\d4r\build\private-textures-gfx1201"`
|
||||
to the game command to install it. The object/manifest hashes and DLSS identity
|
||||
must match validation. These generated objects contain NVIDIA code, remain
|
||||
private, and are excluded from the distributable package. Ordinary launches
|
||||
inherit the game's resolution/window settings; the default argument is `-dx12`.
|
||||
|
||||
Original files are backed up before replacement. Restore with:
|
||||
|
||||
```powershell
|
||||
|
||||
@@ -73,5 +73,70 @@ Native FP8 is not the validated baseline and has no performance claim here.
|
||||
Replacing the exact per-MMA f16 rounding with f32 FP8 accumulation requires a
|
||||
new layer/reference validation; it is not justified by the observed CPU stalls.
|
||||
|
||||
Further game coverage (HIP kernel profiling disabled, GPU output checks enabled):
|
||||
M at 1920x1080 completes 4833 frames, all finite. K at 3840x2160 completes
|
||||
4685 frames, all finite. Neither has a backend failure, CPU image copy or an
|
||||
aged-frame fallback. Their mean interop/NGX CPU stages total about 13.6 and
|
||||
10.1 ms respectively; these are different scenes, not a K/M comparison.
|
||||
|
||||
The no-debugger K/4K run completes 5774 frames, with 5774 finite output scans
|
||||
and 63514 native transformer launches. The user reports 38-42 FPS. Additional
|
||||
CPU measurements exclude replay/publication as the main cause of low FPS:
|
||||
|
||||
| CPU measurement | Mean, ms | Median, ms |
|
||||
| --- | ---: | ---: |
|
||||
| Command split setup | 0.284 | 0.275 |
|
||||
| State replay within split setup | 0.025 | 0.023 |
|
||||
| Submitted texture state publication | 0.029 | 0.029 |
|
||||
| Consumer suffix completion | 0.405 | 0.409 |
|
||||
| NGX recording interval | 27.474 | 28.466 |
|
||||
|
||||
Recording intervals are between NGX recording calls on the same thread, not
|
||||
DXGI Present times. The interval includes rendering and application scheduling.
|
||||
A second K/4K run with the attached debugger completes 3567 frames without
|
||||
failure, with median recording interval 30.091 ms. The runs include different
|
||||
menu/game states; this does not establish a precise debugger FPS speedup.
|
||||
Its sparse debugger events also rule out a per-kernel OutputDebugString storm.
|
||||
An earlier control that stayed on the startup screen completed zero DLSS frames
|
||||
and is excluded. Normal game launches now omit the attached debugger; crash
|
||||
diagnostics remain available with `-CaptureExceptions`.
|
||||
|
||||
The 2961-frame 4K K profile identifies the translated output tail as its largest
|
||||
kernel: 4.401670 ms mean GPU time, versus about 2.875 ms for all eleven native
|
||||
transformer layers combined. Ported upstream's texture-store recipe to native
|
||||
Windows. It retains accuracy mode, wave32, FP16 subnormals and f16 reference
|
||||
accumulation; native FP8 remains disabled.
|
||||
|
||||
Both output variants execute on every checked frame and match the previous
|
||||
RGB output bit-exactly: eight consecutive 512x288 frames per variant, then
|
||||
three consecutive 3840x2160 frames per variant. All RGBA components are finite.
|
||||
Identical synthetic inputs and public NGX flags give these 4K event times
|
||||
(three samples each; host initialization and readback are outside the event):
|
||||
|
||||
| K output variant | Translated mean, ms | Native stores mean, ms |
|
||||
| --- | ---: | ---: |
|
||||
| LDR, regular depth, high-resolution MV | 2.907823 | 1.803233 |
|
||||
| HDR, inverted depth, low-resolution MV | 2.978547 | 2.088347 |
|
||||
|
||||
A subsequent game run executes 6221 frames with the native HDR output tail,
|
||||
zero backend failures and mean tail GPU time 1.801918 ms. The earlier game
|
||||
profile's 4.401670 ms is from a different scene/scheduling state; use the
|
||||
identical-input table for the controlled comparison, not an inferred FPS
|
||||
multiplier. This profiled game run did not enable the NaN scan.
|
||||
|
||||
Reproduce the private texture build and full-frame comparisons:
|
||||
|
||||
```powershell
|
||||
.\scripts\windows\build-native-texture.ps1 -DlssDll "$PWD\nvngx_dlss.dll"
|
||||
.\scripts\windows\build-native-texture.ps1 -DlssDll "$PWD\nvngx_dlss.dll" -Kernel hiluma_engine_output_depthreg_mvhi_ldr_max_v2_rel
|
||||
.\scripts\windows\test-native-texture.ps1 -OutputResolution 3840x2160 -Iterations 3
|
||||
```
|
||||
|
||||
These objects contain code derived from the locally supplied NVIDIA PTX. They
|
||||
stay in ignored local directories and are excluded from the public package.
|
||||
The installer accepts them with `-LocalTextureKernels` only after validation,
|
||||
checking the exact DLL identity, object and manifest hashes. M retains its
|
||||
independently validated FP16-equivalent baseline.
|
||||
|
||||
References: [HIP events](https://rocmdocs.amd.com/projects/HIP/en/develop/doxygen/html/group___event.html),
|
||||
[HIP occupancy API](https://rocm.docs.amd.com/projects/HIP/en/latest/doxygen/html/group___occupancy.html).
|
||||
@@ -9,6 +9,51 @@ validation coverage are recorded below. Performance remains in progress.
|
||||
|
||||
## Milestone and gates
|
||||
|
||||
Upstream refreshed and merged on 2026-10-01 through
|
||||
[`f0d1a65`](https://github.com/countervolts/d4r/commit/f0d1a65e27aff6cbe7eeaa13227c80facb9056ab)
|
||||
and artifact fix `65dfc9f`. Windows retains strict f16 rounding and the exact
|
||||
M attention baseline. Upstream's optional denormal override now drops only
|
||||
FP32 requirements, preserving FP16/FP64. All sixteen rebuilt native K/M
|
||||
objects have the same SHA256 as the previously validated objects. All thirteen
|
||||
ZLUDA patches apply to the pinned fresh base; all sixteen CTest gates pass.
|
||||
The local ZLUDA source includes that denormal correction at `2378af7`; the
|
||||
measured runtime remains the separately recorded `b0161a4` binary, with the
|
||||
override unset. Package metadata records the binary build's actual commit.
|
||||
|
||||
Additional production-style runs: M at 1920x1080 completes 4833 frames with
|
||||
4833 finite-output GPU scans and no backend failures. K at 3840x2160 completes
|
||||
4685 frames, 51535 native launches and 4685 finite-output GPU scans. Both use
|
||||
current-frame VRAM interop, no CPU image copy and no previous-frame fallback.
|
||||
The user confirms visible imagery at 4K but reports low FPS and about 25% GPU
|
||||
usage. This percentage is not an isolated DLSS/WMMA hardware measurement.
|
||||
|
||||
The game runner now launches without an attached debugger by default, while
|
||||
retaining a kill-on-close process job, raw stdout/stderr, exit code, loaded
|
||||
module inventory and frontend/driver logs. `-CaptureExceptions` enables the
|
||||
original attached debugger and minidumps. A K/4K run without it completes
|
||||
5774 frames / 63514 native launches / 5774 finite-output scans, zero failures.
|
||||
The user reports 38-42 FPS. Recording/replay measurements: average split setup
|
||||
0.284 ms, replay 0.025 ms, submitted texture-state publication 0.029 ms,
|
||||
suffix completion 0.405 ms. These do not explain tens of milliseconds of
|
||||
missing performance. NGX recording intervals average 27.474 ms, median
|
||||
28.466 ms; they are not a DXGI Present measurement. The first debugger control
|
||||
never left the startup screen and is explicitly not a valid FPS comparison.
|
||||
|
||||
K output bottleneck: the 2961-frame 4K profile measures its translated output
|
||||
tail at 4.401670 ms, larger than all eleven native transformer layers combined.
|
||||
The Windows texture build preserves upstream Accuracy mode and creates private
|
||||
NVIDIA-derived output objects. Both LDR and the game's HDR/inverted-depth/low-MV
|
||||
variants pass strict full-frame comparisons: eight 512x288 frames each, then
|
||||
three 3840x2160 frames each, max absolute/relative RGB error 0 and all RGBA
|
||||
components finite. Controlled 4K tail GPU means: LDR 2.907823 -> 1.803233 ms;
|
||||
HDR 2.978547 -> 2.088347 ms. The 6221-frame game profile executes the native tail
|
||||
every frame, with zero backend failures and mean tail time 1.801918 ms. Scene
|
||||
differences preclude an exact game FPS speedup claim. `-LocalTextureKernels`
|
||||
requires matching source DLL, object and manifest hashes after validation;
|
||||
public ZIPs exclude these objects. Install/restore with private overrides
|
||||
restores sentinel DLL/config byte-exactly, without duplicate slash/backslash
|
||||
manifest paths.
|
||||
|
||||
Performance diagnostics (2026-10-01): the separate ZLUDA `b0161a4` runtime
|
||||
passes 32 PTX launches in each context and all four native K/M frames with
|
||||
bit-exact RGB against their controls. All thirteen exported patches (0002
|
||||
@@ -141,7 +186,7 @@ Do not integrate NGX until the integer PTX workload is stable on the real GPU.
|
||||
| M6: K layers, full transformer and image validation | PASS: 11 real-weight layers, 44 native launches, four D3D12 frames bit-exact against native CUDA, 81.85–92.91 dB versus translated K |
|
||||
| M7: M FP16-equivalent baseline and full transformer | PASS: all blocks of 40 native launches exactly match NumPy; targeted independent PTX regression matches; four D3D12 frames bit-exact against corrected translated M |
|
||||
| M8: standalone Windows D3D12 NGX harness | PASS for ordinary recorded EvaluateFeature and explicit boundaries: K/M, four frames each, no shim CPU image copies; both bit-exact against explicit boundary baselines |
|
||||
| M9: OptiScaler integration, profiling, installation | Patched frontend K/M PASS; K game menu and M saved level PASS; profiling and final package pending |
|
||||
| M9: OptiScaler integration, profiling, installation | Patched frontend and game K/M PASS, including K 4K and M 1080p finite output; native/translated GPU profiling and reversible install PASS; remaining FPS investigation and final archive in progress |
|
||||
|
||||
## Public NGX identity and OptiScaler gate (2026-10-01)
|
||||
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory=$true)][string]$DlssDll,
|
||||
[string]$Kernel = 'hiluma_engine_output_depthinv_mvlo_hdr_max_v2_rel',
|
||||
[string]$HipRoot = 'C:\Program Files\AMD\ROCm\7.2',
|
||||
[string]$ZludaRoot,
|
||||
[string]$OutputDirectory
|
||||
)
|
||||
$ErrorActionPreference='Stop'
|
||||
$repo=[IO.Path]::GetFullPath((Join-Path $PSScriptRoot '../..'))
|
||||
if (!$ZludaRoot) { $ZludaRoot=Join-Path $repo 'dist/zluda-windows-profile' }
|
||||
if (!$OutputDirectory) { $OutputDirectory=Join-Path $repo 'build/private-textures-gfx1201' }
|
||||
& python (Join-Path $repo 'tools/windows/build_native_texture.py') --dlss-dll $DlssDll --hip-root $HipRoot `
|
||||
--zluda-root $ZludaRoot --kernel $Kernel --output-directory $OutputDirectory
|
||||
if ($LASTEXITCODE) { throw "Native texture build failed ($LASTEXITCODE); inspect the output work directory." }
|
||||
@@ -6,7 +6,7 @@ param(
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$repo = [IO.Path]::GetFullPath((Join-Path $PSScriptRoot '../..'))
|
||||
$defaults = @{PackageRoot='dist/windows-rdna4-game'; DiagnosticRoot='dist/windows-rdna4-command-list';
|
||||
OptiScalerRoot='dist/optiscaler-windows-d4r'; ZludaRoot='dist/zluda-windows-f16-reference';
|
||||
OptiScalerRoot='dist/optiscaler-windows-d4r'; ZludaRoot='dist/zluda-windows-profile';
|
||||
HipRoot='.tools/therock-10.2.0a20260929/_rocm_sdk_core';
|
||||
KernelKRoot='build/native-k-gfx1201'; KernelMRoot='build/native-m-gfx1201'}
|
||||
foreach ($name in $defaults.Keys) {
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
[CmdletBinding()]
|
||||
param([string]$TextureRoot, [string]$OutputDirectory, [int]$Iterations=8, [string]$OutputResolution='512x288')
|
||||
$ErrorActionPreference='Stop'
|
||||
$repo=[IO.Path]::GetFullPath((Join-Path $PSScriptRoot '../..'))
|
||||
if (!$TextureRoot) { $TextureRoot=Join-Path $repo 'build/private-textures-gfx1201' }
|
||||
if (!$OutputDirectory) { $OutputDirectory=Join-Path $repo 'test-results/k-native-texture-reference' }
|
||||
if ($OutputResolution -notmatch '^([0-9]+)x([0-9]+)$') { throw 'OutputResolution must be WIDTHxHEIGHT.' }
|
||||
$outputWidth=[int]$Matches[1]; $outputHeight=[int]$Matches[2]
|
||||
$combined=Join-Path $OutputDirectory 'private-native'
|
||||
New-Item -ItemType Directory -Force $combined | Out-Null
|
||||
foreach ($directory in @((Join-Path $repo 'build/native-k-gfx1201'),$TextureRoot)) {
|
||||
Get-ChildItem -LiteralPath $directory -Filter '*.hsaco' | Copy-Item -Destination $combined -Force
|
||||
}
|
||||
& python (Join-Path $repo 'kernels/tools/kernel_manifest.py') $combined (Join-Path $repo 'nvngx_dlss.dll')
|
||||
if ($LASTEXITCODE) { throw 'Combined private manifest failed.' }
|
||||
$settings=@{D4R_VALIDATE_OUTPUT='1'; D4R_PROFILE_STAGES='1'; D4R_ZLUDA_PROFILE='1'; D4R_QUIET_API='1';
|
||||
ZLUDA_CACHE_DIR=(Join-Path $repo 'build/zluda-cache-windows'); PYTHONPATH=(Join-Path $repo '.tools/python/vendor');
|
||||
D4R_ZLUDA_NATIVE_DIR=$null}
|
||||
$old=@{}
|
||||
try {
|
||||
foreach ($key in $settings.Keys) { $old[$key]=[Environment]::GetEnvironmentVariable($key,'Process'); [Environment]::SetEnvironmentVariable($key,$settings[$key],'Process') }
|
||||
foreach ($flags in @(0,11)) {
|
||||
foreach ($mode in @('control','candidate')) {
|
||||
$env:D4R_ZLUDA_NATIVE_DIR=if ($mode -eq 'control') { Join-Path $repo 'build/native-k-gfx1201' } else { $combined }
|
||||
$arguments=@{RuntimeProfile='therock'; ZludaRoot=(Join-Path $repo 'dist/zluda-windows-profile');
|
||||
PackageRoot=(Join-Path $repo 'dist/windows-rdna4-command-list'); NgxCore=(Join-Path $repo '_nvngx.dll');
|
||||
DlssDll=(Join-Path $repo 'nvngx_dlss.dll'); NgxMode='d3d12'; NgxAbi='project-legacy'; Preset=11;
|
||||
NgxCreateFlags=$flags; NgxOutputResolution=$OutputResolution; NgxOnly=$true; RequireNativeNetwork=$true; CommandListBackend=$true;
|
||||
PixelProfile='depth-stencil'; BarrierMode='inherited-legacy'; CaptureExceptions=$true; EarlyIndirectProbe=$true;
|
||||
Iterations=$Iterations; TimeoutSeconds=300; OptiScalerDll=(Join-Path $repo 'dist/optiscaler-windows-d4r/OptiScaler.dll');
|
||||
OutputDirectory=(Join-Path $OutputDirectory "$mode-$flags")}
|
||||
& (Join-Path $PSScriptRoot 'test-windows-rdna4.ps1') @arguments
|
||||
if ($LASTEXITCODE) { throw "Texture $mode flags=$flags workload failed." }
|
||||
& python (Join-Path $repo 'tools/windows/profile_report.py') $arguments.OutputDirectory --output (Join-Path $arguments.OutputDirectory 'profile.json')
|
||||
if ($LASTEXITCODE) { throw 'Texture profile report failed.' }
|
||||
}
|
||||
$control=Join-Path $OutputDirectory "control-$flags"; $candidate=Join-Path $OutputDirectory "candidate-$flags"
|
||||
$kernel=if ($flags -eq 11) { 'hiluma_engine_output_depthinv_mvlo_hdr_max_v2_rel' } else { 'hiluma_engine_output_depthreg_mvhi_ldr_max_v2_rel' }
|
||||
$stderr=Get-Content -LiteralPath (Join-Path $candidate 'd3d12-evaluate-preset-11.stderr.log') -Raw
|
||||
$hits=[regex]::Matches($stderr, ('\[d4r-launch\] kernel="' + $kernel + '" backend=native')).Count
|
||||
if ($hits -ne $Iterations) { throw "Texture override was not executed on every frame: $kernel hits=$hits" }
|
||||
& python (Join-Path $repo 'tools/windows/frame_compare.py') --reference $control --actual $candidate --width $outputWidth --height $outputHeight --exact
|
||||
if ($LASTEXITCODE) { throw "Texture flags=$flags full-frame RGB mismatch; preserve baseline." }
|
||||
}
|
||||
@{passed=$true; architecture='gfx1201'; framesPerVariant=$Iterations; outputResolution=$OutputResolution; createFlags=@(0,11); strictRgb=$true;
|
||||
manifestSha256=(Get-FileHash -LiteralPath (Join-Path $TextureRoot 'd4r-kernels.txt') -Algorithm SHA256).Hash.ToLowerInvariant();
|
||||
privateNvidiaDerivedCode=$true; source=(Get-Content -LiteralPath (Join-Path $TextureRoot 'build-info.json') -Raw | ConvertFrom-Json)} |
|
||||
ConvertTo-Json -Depth 12 | Set-Content -LiteralPath (Join-Path $TextureRoot 'validation.json') -Encoding UTF8
|
||||
Write-Host "PASS: both private K output variants executed and matched every RGB component. $OutputDirectory"
|
||||
} finally {
|
||||
foreach ($key in $old.Keys) { [Environment]::SetEnvironmentVariable($key,$old[$key],'Process') }
|
||||
}
|
||||
@@ -11,6 +11,8 @@ param(
|
||||
[ValidateSet('driver','sdk','project','project-legacy')][string]$NgxAbi = 'driver',
|
||||
[string]$OptiScalerDll,
|
||||
[ValidateSet(5, 11, 13)][int]$Preset = 11,
|
||||
[ValidateSet(0,11)][int]$NgxCreateFlags = 0,
|
||||
[string]$NgxOutputResolution = '512x288',
|
||||
[switch]$NgxOnly,
|
||||
[switch]$Trace,
|
||||
[switch]$RequireNativeNetwork,
|
||||
@@ -311,7 +313,8 @@ DisableSplash=true
|
||||
$summary.optiScaler = @{path=$OptiScalerDll; mode='standalone'; preset=$Preset}
|
||||
$ngxArguments += @('--ngx-frontend', 'optiscaler')
|
||||
}
|
||||
$ngxArguments += @('--module', $localShim, '--pixel-profile', $PixelProfile, '--barrier-mode', $BarrierMode)
|
||||
$ngxArguments += @('--module', $localShim, '--pixel-profile', $PixelProfile, '--barrier-mode', $BarrierMode,
|
||||
'--ngx-create-flags', "$NgxCreateFlags", '--ngx-output-resolution', $NgxOutputResolution)
|
||||
if ($EarlyIndirectProbe) {
|
||||
if (!$CommandListBackend) { throw '-EarlyIndirectProbe requires -CommandListBackend' }
|
||||
$ngxArguments += @('--early-indirect', '1')
|
||||
@@ -366,7 +369,14 @@ DisableSplash=true
|
||||
[IO.Path]::GetFileName($resolvedRuntime) -notmatch '^d4r-ngx-[0-9a-f]{32}$') {
|
||||
throw 'Refusing to remove a path outside the private NGX temporary runtime.'
|
||||
}
|
||||
Remove-Item -LiteralPath $resolvedRuntime -Recurse -Force
|
||||
# A scanner can briefly hold the just-unloaded DLL open. Cleanup must
|
||||
# not skip environment restoration or discard the completed GPU logs.
|
||||
$cleanupError=$null
|
||||
foreach ($attempt in 1..10) {
|
||||
try { Remove-Item -LiteralPath $resolvedRuntime -Recurse -Force -ErrorAction Stop; $cleanupError=$null; break }
|
||||
catch { $cleanupError=$_.Exception.Message; if ($attempt -lt 10) { Start-Sleep -Milliseconds 300 } }
|
||||
}
|
||||
if ($cleanupError) { $summary.cleanupError=$cleanupError; $exitStatus=1; Write-Warning "Private runtime cleanup failed: $cleanupError" }
|
||||
}
|
||||
$summary.passed = $exitStatus -eq 0
|
||||
$summary | ConvertTo-Json -Depth 8 | Set-Content -LiteralPath (Join-Path $OutputDirectory 'summary.json') -Encoding UTF8
|
||||
|
||||
@@ -7,10 +7,13 @@ param(
|
||||
[int]$RunSeconds = 0,
|
||||
[string]$DiagnosticResolution,
|
||||
[string]$CacheDirectory = $env:ZLUDA_CACHE_DIR,
|
||||
[string]$LocalTextureKernels,
|
||||
[switch]$ValidateOutput,
|
||||
[switch]$ProfileStages,
|
||||
[switch]$ProfileKernels,
|
||||
[switch]$VerboseRuntime,
|
||||
[string[]]$GameArguments = @('-dx12','-windowed','-ResX=1280','-ResY=720'),
|
||||
[switch]$CaptureExceptions,
|
||||
[string[]]$GameArguments = @('-dx12'),
|
||||
[string]$OutputDirectory
|
||||
)
|
||||
$ErrorActionPreference = 'Stop'
|
||||
@@ -56,13 +59,40 @@ foreach ($file in $metadata.files) {
|
||||
if ((Get-FileHash -LiteralPath (Join-Path $package $file.path) -Algorithm SHA256).Hash -ne $file.sha256) { throw "Package file changed; rebuild package: $($file.path)" }
|
||||
}
|
||||
if ((Get-FileHash -LiteralPath $DlssDll -Algorithm SHA256).Hash -ne $metadata.dlssSha256) { throw 'DLSS DLL does not match the validated 310.9.1 native manifest. Rebuild and validate native manifests for this DLL first.' }
|
||||
$textureFiles=@(); $textureManifest=@()
|
||||
if ($LocalTextureKernels) {
|
||||
$LocalTextureKernels=(Get-Item -LiteralPath $LocalTextureKernels -ErrorAction Stop).FullName
|
||||
$validation=Get-Content -LiteralPath (Join-Path $LocalTextureKernels 'validation.json') -Raw | ConvertFrom-Json
|
||||
if (!$validation.passed -or !$validation.strictRgb -or $validation.architecture -ne 'gfx1201' -or
|
||||
$validation.source.dlssSha256 -ne $metadata.dlssSha256 -or !$validation.source.accuracy) {
|
||||
throw 'Local texture kernels must pass test-native-texture.ps1 with the exact packaged DLSS identity.'
|
||||
}
|
||||
foreach ($object in $validation.source.objects.PSObject.Properties) {
|
||||
if ($object.Name -notin @('hiluma_engine_output_depthinv_mvlo_hdr_max_v2_rel.hsaco','hiluma_engine_output_depthreg_mvhi_ldr_max_v2_rel.hsaco')) {
|
||||
throw "Unvalidated local texture variant: $($object.Name)"
|
||||
}
|
||||
$path=Join-Path $LocalTextureKernels $object.Name
|
||||
if ((Get-FileHash -LiteralPath $path -Algorithm SHA256).Hash.ToLowerInvariant() -ne $object.Value) { throw 'Local texture object changed after validation.' }
|
||||
$textureFiles+=Get-Item -LiteralPath $path
|
||||
}
|
||||
if ($textureFiles.Count -ne 2) { throw 'Expected both validated local K output variants.' }
|
||||
# Use the manifest covered by validation, rather than a mutable extra file.
|
||||
foreach ($line in Get-Content -LiteralPath (Join-Path $LocalTextureKernels 'd4r-kernels.txt')) {
|
||||
if (!$line.Trim() -or $line.StartsWith('#')) { continue }
|
||||
if ($line -notmatch '^([A-Za-z0-9_]+) ([0-9a-f]{16})$' -or ($Matches[1]+'.hsaco') -notin @($textureFiles.Name)) { throw 'Invalid local texture manifest.' }
|
||||
$textureManifest+=$line
|
||||
}
|
||||
if ($textureManifest.Count -ne 2) { throw 'Expected two local texture manifest identities.' }
|
||||
if ((Get-FileHash -LiteralPath (Join-Path $LocalTextureKernels 'd4r-kernels.txt') -Algorithm SHA256).Hash.ToLowerInvariant() -ne $validation.manifestSha256) { throw 'Local texture manifest changed after validation.' }
|
||||
}
|
||||
$manifest = if (Test-Path -LiteralPath $manifestPath) { Get-Content -LiteralPath $manifestPath -Raw | ConvertFrom-Json } else { [pscustomobject]@{game=$GameExe; files=@()} }
|
||||
if ($manifest.game -ne $GameExe) { throw 'Backup manifest belongs to another executable.' }
|
||||
if (!(Test-Path -LiteralPath $manifestPath) -and (Test-Path -LiteralPath (GamePath 'd4r'))) { throw 'An existing d4r directory has no installation manifest; preserved.' }
|
||||
New-Item -ItemType Directory -Force $backup | Out-Null
|
||||
function InstallFile([string]$source, [string]$relative) {
|
||||
$relative=$relative.Replace('/','\')
|
||||
$path = GamePath $relative
|
||||
$entry = @($manifest.files | Where-Object path -eq $relative) | Select-Object -First 1
|
||||
$entry = @($manifest.files | Where-Object { $_.path.Replace('/','\') -eq $relative }) | Select-Object -First 1
|
||||
if ($entry) {
|
||||
if ((Test-Path -LiteralPath $path) -and (Get-FileHash -LiteralPath $path -Algorithm SHA256).Hash -ne $entry.installedHash) { throw "Installed file changed; preserved: $path" }
|
||||
} else {
|
||||
@@ -82,6 +112,13 @@ function InstallFile([string]$source, [string]$relative) {
|
||||
}
|
||||
InstallFile (Join-Path $package 'OptiScaler.dll') 'dxgi.dll'
|
||||
foreach ($file in Get-ChildItem -LiteralPath (Join-Path $package 'd4r') -Recurse -File) { InstallFile $file.FullName $file.FullName.Substring($package.Length+1) }
|
||||
if ($LocalTextureKernels) {
|
||||
foreach ($file in $textureFiles) { InstallFile $file.FullName ('d4r/native/' + $file.Name) }
|
||||
$combinedManifest=Join-Path $backup 'generated-native-manifest.txt'
|
||||
@(Get-Content -LiteralPath (Join-Path $package 'd4r/native/d4r-kernels.txt')) + $textureManifest |
|
||||
Set-Content -LiteralPath $combinedManifest -Encoding ASCII
|
||||
InstallFile $combinedManifest 'd4r/native/d4r-kernels.txt'
|
||||
}
|
||||
# Private local test copies only; these are never added to the public package.
|
||||
InstallFile $NgxCore 'd4r/vendor/_nvngx.dll'
|
||||
InstallFile $DlssDll 'd4r/vendor/nvngx_dlss.dll'
|
||||
@@ -136,13 +173,14 @@ $settings = @{
|
||||
D4R_ZLUDA_IGNORE_DENORMAL=$null; D4R_ZLUDA_FAST_MATH=$null;
|
||||
D4R_ZLUDA_WMMA_F32ACC=$null; D4R_ZLUDA_WAVE64=$null;
|
||||
D4R_QUIET_API=$(if ($VerboseRuntime) { $null } else { '1' });
|
||||
D4R_VALIDATE_OUTPUT=$(if ($ValidateOutput) { '1' } else { $null });
|
||||
D4R_PROFILE_STAGES=$(if ($ProfileStages) { '1' } else { $null });
|
||||
D4R_ZLUDA_PROFILE=$(if ($ProfileKernels) { '1' } else { $null });
|
||||
D4R_DIAG_DIR=$OutputDirectory; ZLUDA_LOG_DIR=(Join-Path $OutputDirectory 'zluda-trace');
|
||||
ZLUDA_CACHE_DIR=$CacheDirectory; PATH=((GamePath 'd4r/hip/bin') + ';' + (GamePath 'd4r/zluda') + ';' + $env:PATH)
|
||||
}
|
||||
$old = @{}; $process = $null; $originalSettings = @()
|
||||
$outFile = $null; $errFile = $null; $outTask = $null; $errTask = $null
|
||||
if ($ValidateOutput) { $settings['D4R_VALIDATE_OUTPUT']='1' }
|
||||
if ($ProfileStages) { $settings['D4R_PROFILE_STAGES']='1' }
|
||||
try {
|
||||
if ($DiagnosticResolution) {
|
||||
if ([IO.Path]::GetFileName($GameExe) -ne 'SHProto-Win64-Shipping.exe' -or $DiagnosticResolution -notmatch '^([0-9]{3,4})x([0-9]{3,4})$') { throw 'DiagnosticResolution currently supports Silent Hill 2 only; use e.g. 1280x720.' }
|
||||
@@ -173,10 +211,12 @@ try {
|
||||
Write-Host "Temporary diagnostic resolution: $DiagnosticResolution; original settings restore at exit."
|
||||
}
|
||||
foreach ($key in $settings.Keys) { $old[$key] = [Environment]::GetEnvironmentVariable($key,'Process'); [Environment]::SetEnvironmentVariable($key,$settings[$key],'Process') }
|
||||
@{preset=$Preset; game=$GameExe; package=$metadata; driver=@(Get-CimInstance Win32_VideoController | Select-Object Name,DriverVersion,DriverDate); arguments=$GameArguments} |
|
||||
@{preset=$Preset; game=$GameExe; package=$metadata; attachedDebugger=[bool]$CaptureExceptions; driver=@(Get-CimInstance Win32_VideoController | Select-Object Name,DriverVersion,DriverDate); arguments=$GameArguments} |
|
||||
ConvertTo-Json -Depth 12 | Set-Content -LiteralPath (Join-Path $OutputDirectory 'environment.json') -Encoding UTF8
|
||||
function Quote([string]$value) { if ($value.Contains('"')) { throw 'A command argument contains an unsupported quote.' }; return '"' + $value + '"' }
|
||||
$arguments = @('--output-directory', (Quote $OutputDirectory), '--', (Quote $GameExe)) + @($GameArguments | ForEach-Object { Quote $_ })
|
||||
$arguments = @('--output-directory', (Quote $OutputDirectory))
|
||||
if (!$CaptureExceptions) { $arguments += '--no-debug' }
|
||||
$arguments += @('--', (Quote $GameExe)) + @($GameArguments | ForEach-Object { Quote $_ })
|
||||
# Drain raw bytes concurrently. PowerShell's line-oriented redirection can
|
||||
# throttle a verbose game and distort the very timings we are collecting.
|
||||
$info = [Diagnostics.ProcessStartInfo]::new()
|
||||
@@ -186,7 +226,7 @@ try {
|
||||
$info.UseShellExecute = $false; $info.CreateNoWindow = $true
|
||||
$info.RedirectStandardOutput = $true; $info.RedirectStandardError = $true
|
||||
$process = [Diagnostics.Process]::new(); $process.StartInfo = $info
|
||||
if (!$process.Start()) { throw 'Game debugger launch failed.' }
|
||||
if (!$process.Start()) { throw 'Game diagnostic launch failed.' }
|
||||
$outFile = [IO.File]::Open((Join-Path $OutputDirectory 'd4r.stdout.log'), [IO.FileMode]::Create, [IO.FileAccess]::Write, [IO.FileShare]::Read)
|
||||
$errFile = [IO.File]::Open((Join-Path $OutputDirectory 'd4r.stderr.log'), [IO.FileMode]::Create, [IO.FileAccess]::Write, [IO.FileShare]::Read)
|
||||
$outTask = $process.StandardOutput.BaseStream.CopyToAsync($outFile)
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Build a private gfx1201 texture override from the user's local DLSS DLL.
|
||||
|
||||
The resulting PTX/code object contains NVIDIA code and must remain local.
|
||||
No Wine, shell script, GPU, CUDA SDK or system installation is required.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--dlss-dll', type=pathlib.Path, required=True)
|
||||
parser.add_argument('--hip-root', type=pathlib.Path, required=True)
|
||||
parser.add_argument('--zluda-root', type=pathlib.Path, required=True)
|
||||
parser.add_argument('--kernel', required=True)
|
||||
parser.add_argument('--output-directory', type=pathlib.Path, required=True)
|
||||
args = parser.parse_args()
|
||||
if not re.fullmatch(r'hiluma_engine_output_depth(?:inv|reg)_mv(?:hi|lo)_(?:hdr|ldr)(?:_max)?_v2_rel', args.kernel):
|
||||
parser.error('this initial validated build recipe supports K v2 output variants only')
|
||||
repo = pathlib.Path(__file__).resolve().parents[2]
|
||||
output = args.output_directory.resolve()
|
||||
output.mkdir(parents=True, exist_ok=True)
|
||||
work = output / ('work-' + args.kernel)
|
||||
work.mkdir(exist_ok=True)
|
||||
hip = args.hip_root.resolve()
|
||||
zluda = args.zluda_root.resolve()
|
||||
tools = {name: hip / 'bin' / (name + '.exe') for name in ('clang++', 'llvm-dis', 'llvm-as')}
|
||||
emitter = zluda / 'd4r_emit.exe'
|
||||
for path in [args.dlss_dll, emitter, *tools.values()]:
|
||||
if not path.is_file():
|
||||
parser.error(f'missing dependency: {path}')
|
||||
env = os.environ.copy()
|
||||
env.update(D4R_PREFER_ACCURACY='1', D4R_ZLUDA_WMMA='1', D4R_ZLUDA_WMMA_FP8='1',
|
||||
D4R_ZLUDA_WMMA_FP8_NATIVE='0', D4R_ZLUDA_WMMA_F16_REFERENCE='1',
|
||||
D4R_ZLUDA_IMPLICIT_MAX_BLOCK='256')
|
||||
for name in ('D4R_ZLUDA_IGNORE_DENORMAL', 'D4R_ZLUDA_FAST_MATH', 'D4R_ZLUDA_WMMA_F32ACC',
|
||||
'D4R_ZLUDA_WAVE64', 'D4R_ZLUDA_PROFILE'):
|
||||
env.pop(name, None)
|
||||
env['PATH'] = str(zluda) + os.pathsep + str(hip / 'bin') + os.pathsep + env['PATH']
|
||||
|
||||
def run(label, command):
|
||||
with (work / (label + '.log')).open('wb') as log:
|
||||
result = subprocess.run([str(arg) for arg in command], env=env, stdout=log, stderr=subprocess.STDOUT)
|
||||
if result.returncode:
|
||||
print((work / (label + '.log')).read_text(errors='replace')[-8000:], file=sys.stderr)
|
||||
raise RuntimeError(f'{label} failed ({result.returncode}); see {work}')
|
||||
|
||||
ptx = work / 'ptx'
|
||||
run('extract', [sys.executable, repo / 'kernels/tools/extract_dlss_ptx.py', args.dlss_dll, ptx])
|
||||
env['D4R_DLSS_PTX_DIR'] = str(ptx)
|
||||
edited = work / (args.kernel + '.ptx')
|
||||
run('make-ptx', [sys.executable, repo / 'kernels/tex/make_ptx.py', args.kernel, edited])
|
||||
raw, ir, extra = work / 'raw.bc', work / 'raw.ll', work / 'extra.bc'
|
||||
run('compile-helper', [tools['clang++'], '-x', 'hip', '-std=c++20', '-nogpuinc', '-nogpulib', '-O3',
|
||||
'-mno-wavefrontsize64', '--offload-device-only', '--offload-arch=gfx1201',
|
||||
'-fgpu-rdc', '-emit-llvm', '-c', '-Xclang', '-fdenormal-fp-math=dynamic',
|
||||
'-DD4R_ACCURACY', '-DD4R_KERNEL_NAME=' + args.kernel,
|
||||
'-o', raw, repo / 'kernels/tex/sust_only.hip'])
|
||||
run('disassemble', [tools['llvm-dis'], raw, '-o', ir])
|
||||
lines = []
|
||||
for line in ir.read_text().splitlines():
|
||||
if re.search(r'@llvm.used|wchar_size|llvm.module.flags|__hip_cuid', line):
|
||||
continue
|
||||
line = line.replace('optnone', '').replace('"target-cpu"="gfx1201"', '')
|
||||
lines.append(re.sub(r'"target-features"="[^"]+"', '', line))
|
||||
ir.write_text('\n'.join(lines) + '\n')
|
||||
run('assemble', [tools['llvm-as'], ir, '-o', extra])
|
||||
env['D4R_ZLUDA_EXTRA_BC'] = str(extra)
|
||||
run('emit', [emitter, edited, work / 'emitted', 'gfx1201'])
|
||||
shutil.copyfile(work / 'emitted/module.hsaco', output / (args.kernel + '.hsaco'))
|
||||
run('manifest', [sys.executable, repo / 'kernels/tools/kernel_manifest.py', output, args.dlss_dll])
|
||||
metadata = dict(architecture='gfx1201', accuracy=True, nativeFP8=False,
|
||||
privateNvidiaDerivedCode=True, validation='not yet validated; do not install before comparison',
|
||||
dlssSha256=hashlib.sha256(args.dlss_dll.read_bytes()).hexdigest(),
|
||||
zludaBuild=json.loads((zluda / 'build-info.json').read_text(encoding='utf-8-sig')),
|
||||
compiler=subprocess.check_output([str(tools['clang++']), '--version'], env=env).decode(errors='replace'),
|
||||
objects={p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in output.glob('*.hsaco')})
|
||||
(output / 'build-info.json').write_text(json.dumps(metadata, indent=2) + '\n')
|
||||
print(f'Built PRIVATE gfx1201 output override: {output}; validation required before use')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -6,6 +6,7 @@
|
||||
#include <mutex>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
#include <chrono>
|
||||
|
||||
namespace d4r::win::commands {
|
||||
namespace {
|
||||
@@ -13,6 +14,9 @@ thread_local bool internal = false;
|
||||
std::mutex map_mutex, installation_mutex, error_mutex;
|
||||
std::string last_error;
|
||||
bool installed = false;
|
||||
using Clock = std::chrono::steady_clock;
|
||||
bool profile_commands() { static const bool enabled = std::getenv("D4R_PROFILE_STAGES") != nullptr; return enabled; }
|
||||
double milliseconds(Clock::time_point start) { return std::chrono::duration<double, std::milli>(Clock::now() - start).count(); }
|
||||
struct Event { UINT metadata; std::vector<uint8_t> data; };
|
||||
using State = std::function<void(ID3D12GraphicsCommandList*)>;
|
||||
struct TrackedResource {
|
||||
@@ -385,6 +389,8 @@ void complete(ID3D12CommandQueue* queue) {
|
||||
}
|
||||
void STDMETHODCALLTYPE hook_execute(ID3D12CommandQueue* queue, UINT count, ID3D12CommandList* const* lists) {
|
||||
if (internal) { original_execute(queue, count, lists); return; }
|
||||
const auto submitStart = Clock::now();
|
||||
double publishMs = 0, boundaryMs = 0, drainMs = 0;
|
||||
std::vector<std::shared_ptr<Recording>> owners;
|
||||
std::vector<ID3D12CommandList*> pending;
|
||||
std::vector<ResourceStates> pending_states;
|
||||
@@ -394,7 +400,9 @@ void STDMETHODCALLTYPE hook_execute(ID3D12CommandQueue* queue, UINT count, ID3D1
|
||||
original_execute(queue, UINT(pending.size()), pending.data());
|
||||
// Publish in the actual submitted order, never while recording.
|
||||
// The callback's external fence waits for these producers on GPU.
|
||||
const auto publishStart = Clock::now();
|
||||
for (const auto& states : pending_states) publish_states(states);
|
||||
publishMs += milliseconds(publishStart);
|
||||
pending.clear(); pending_states.clear();
|
||||
}
|
||||
};
|
||||
@@ -416,13 +424,19 @@ void STDMETHODCALLTYPE hook_execute(ID3D12CommandQueue* queue, UINT count, ID3D1
|
||||
owners.push_back(recording); boundaries = true;
|
||||
for (const auto& segment : recording->segments) {
|
||||
pending.push_back(segment.list); pending_states.push_back(segment.resource_states); flush();
|
||||
const auto boundaryStart = Clock::now();
|
||||
segment.boundary(queue);
|
||||
boundaryMs += milliseconds(boundaryStart);
|
||||
}
|
||||
pending.push_back(recording->current.Get());
|
||||
pending_states.push_back(recording->resource_states);
|
||||
}
|
||||
flush();
|
||||
if (boundaries) complete(queue);
|
||||
if (boundaries) {
|
||||
const auto drainStart = Clock::now(); complete(queue); drainMs = milliseconds(drainStart);
|
||||
if (profile_commands()) std::printf("D4R_COMMAND_SUBMIT total_ms=%.3f boundary_ms=%.3f publish_ms=%.3f suffix_drain_ms=%.3f\n",
|
||||
milliseconds(submitStart), boundaryMs, publishMs, drainMs);
|
||||
}
|
||||
} catch (const std::exception& failure) {
|
||||
{ std::lock_guard<std::mutex> lock(error_mutex); last_error = failure.what(); }
|
||||
std::fprintf(stderr, "D4R_D3D12_SUBMIT_FAILURE %s\n", failure.what());
|
||||
@@ -510,6 +524,7 @@ void install(ID3D12Device* device) {
|
||||
}
|
||||
}
|
||||
void record_boundary(ID3D12GraphicsCommandList* list, Boundary callback) {
|
||||
const auto recordStart = Clock::now();
|
||||
if (!installed || !list || !callback) throw std::runtime_error("D3D12 command backend is not initialized");
|
||||
if (!error().empty()) throw std::runtime_error(error());
|
||||
if (list->GetType() != D3D12_COMMAND_LIST_TYPE_DIRECT) throw std::runtime_error("DLSS requires a direct command list");
|
||||
@@ -527,7 +542,9 @@ void record_boundary(ID3D12GraphicsCommandList* list, Boundary callback) {
|
||||
ComPtr<ID3D12CommandAllocator> nextAllocator; ComPtr<ID3D12GraphicsCommandList> next;
|
||||
dx(device->CreateCommandAllocator(D3D12_COMMAND_LIST_TYPE_DIRECT, IID_PPV_ARGS(nextAllocator.GetAddressOf())), "Suffix allocator");
|
||||
dx(device->CreateCommandList(0, D3D12_COMMAND_LIST_TYPE_DIRECT, nextAllocator.Get(), nullptr, IID_PPV_ARGS(next.GetAddressOf())), "Suffix command list");
|
||||
const auto replayStart = Clock::now();
|
||||
for (auto& state : recording.state) state(next.Get());
|
||||
const auto replayMs = milliseconds(replayStart);
|
||||
recording.segments.reserve(recording.segments.size() + 1);
|
||||
for (const auto& event : recording.events) original_BeginEvent(next.Get(), event.metadata, event.data.data(), UINT(event.data.size()));
|
||||
for (size_t n = 0; n < recording.events.size(); ++n) original_EndEvent(segment.list);
|
||||
@@ -537,5 +554,14 @@ void record_boundary(ID3D12GraphicsCommandList* list, Boundary callback) {
|
||||
recording.current_allocator = std::move(nextAllocator);
|
||||
recording.current = std::move(next);
|
||||
std::printf("D4R_D3D12_RECORD boundary=%zu state_calls=%zu frame_age=0\n", recording.segments.size(), recording.state.size());
|
||||
if (profile_commands()) {
|
||||
// This is the interval between NGX recording calls on this thread,
|
||||
// not a Present/FPS measurement or an extra queued frame.
|
||||
thread_local Clock::time_point previous{};
|
||||
const double interval = previous == Clock::time_point{} ? 0 : std::chrono::duration<double, std::milli>(recordStart - previous).count();
|
||||
previous = recordStart;
|
||||
std::printf("D4R_COMMAND_RECORD total_ms=%.3f replay_ms=%.3f state_calls=%zu interval_ms=%.3f\n",
|
||||
milliseconds(recordStart), replayMs, recording.state.size(), interval);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -73,7 +73,9 @@ int main(int argc, char** argv) {
|
||||
ComPtr<ID3D12GraphicsCommandList> list;
|
||||
dx(device->CreateCommandAllocator(D3D12_COMMAND_LIST_TYPE_DIRECT, IID_PPV_ARGS(allocator.GetAddressOf())), "CreateCommandAllocator");
|
||||
dx(device->CreateCommandList(0, D3D12_COMMAND_LIST_TYPE_DIRECT, allocator.Get(), nullptr, IID_PPV_ARGS(list.GetAddressOf())), "CreateCommandList");
|
||||
constexpr unsigned width = 256, height = 144, outWidth = 512, outHeight = 288;
|
||||
const unsigned outWidth = args.output_width, outHeight = args.output_height;
|
||||
const unsigned width = outWidth / 2, height = outHeight / 2;
|
||||
std::printf("D3D12_NGX_DIMENSIONS input=%ux%u output=%ux%u\n", width, height, outWidth, outHeight);
|
||||
struct Texture { ComPtr<ID3D12Resource> image, upload; D3D12_PLACED_SUBRESOURCE_FOOTPRINT footprint{}; UINT64 bytes; } textures[5];
|
||||
for (unsigned i = 0; i < 5; ++i) {
|
||||
auto& texture = textures[i];
|
||||
@@ -230,7 +232,8 @@ int main(int argc, char** argv) {
|
||||
for (const auto& pair : {std::make_pair("Width", width), std::make_pair("Height", height),
|
||||
std::make_pair("OutWidth", outWidth), std::make_pair("OutHeight", outHeight),
|
||||
std::make_pair("CreationNodeMask", 1u), std::make_pair("VisibilityNodeMask", 1u)}) d4r_ngx_set_uint(parameters, pair.first, pair.second);
|
||||
d4r_ngx_set_int(parameters, "PerfQualityValue", 2); d4r_ngx_set_int(parameters, "DLSS.Feature.Create.Flags", 0);
|
||||
d4r_ngx_set_int(parameters, "PerfQualityValue", 2); d4r_ngx_set_int(parameters, "DLSS.Feature.Create.Flags", args.ngx_create_flags);
|
||||
std::printf("D3D12_NGX_CREATE_FLAGS value=%u\n", args.ngx_create_flags);
|
||||
for (const char* name : {"DLSS.Hint.Render.Preset.DLAA", "DLSS.Hint.Render.Preset.Quality", "DLSS.Hint.Render.Preset.Balanced",
|
||||
"DLSS.Hint.Render.Preset.Performance", "DLSS.Hint.Render.Preset.UltraPerformance", "DLSS.Hint.Render.Preset.UltraQuality"}) d4r_ngx_set_uint(parameters, name, args.preset);
|
||||
void* handle = nullptr;
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
#include "diagnostic.h"
|
||||
#include <map>
|
||||
#include <fstream>
|
||||
#include <tlhelp32.h>
|
||||
|
||||
namespace {
|
||||
std::wstring quote(const std::wstring& value) {
|
||||
@@ -31,14 +32,16 @@ uint32_t module_size(HANDLE process, uintptr_t base) {
|
||||
}
|
||||
int main(int argc, char** argv) {
|
||||
try {
|
||||
if (argc < 5 || std::strcmp(argv[1], "--output-directory") || std::strcmp(argv[3], "--"))
|
||||
throw std::runtime_error("Usage: d4r_debug_launcher --output-directory PATH -- EXE [arguments]");
|
||||
const bool debug = argc < 4 || std::strcmp(argv[3], "--no-debug");
|
||||
const int separator = debug ? 3 : 4;
|
||||
if (argc <= separator + 1 || std::strcmp(argv[1], "--output-directory") || std::strcmp(argv[separator], "--"))
|
||||
throw std::runtime_error("Usage: d4r_debug_launcher --output-directory PATH [--no-debug] -- EXE [arguments]");
|
||||
const std::filesystem::path directory(d4r::diag::wide(argv[2]));
|
||||
std::filesystem::create_directories(directory);
|
||||
std::ofstream log(directory / L"debugger.log", std::ios::app);
|
||||
const auto executable = d4r::diag::wide(argv[4]);
|
||||
const auto executable = d4r::diag::wide(argv[separator + 1]);
|
||||
std::wstring command;
|
||||
for (int i = 4; i < argc; ++i) { if (i > 4) command += L' '; command += quote(d4r::diag::wide(argv[i])); }
|
||||
for (int i = separator + 1; i < argc; ++i) { if (i > separator + 1) command += L' '; command += quote(d4r::diag::wide(argv[i])); }
|
||||
STARTUPINFOW startup{sizeof(startup)}; PROCESS_INFORMATION child{};
|
||||
startup.dwFlags = STARTF_USESTDHANDLES;
|
||||
startup.hStdInput = GetStdHandle(STD_INPUT_HANDLE);
|
||||
@@ -50,12 +53,39 @@ int main(int argc, char** argv) {
|
||||
if (!job || !SetInformationJobObject(job, JobObjectExtendedLimitInformation, &limits, sizeof(limits)))
|
||||
throw std::runtime_error("Cannot create diagnostic process lifetime job");
|
||||
if (!CreateProcessW(executable.c_str(), command.data(), nullptr, nullptr, TRUE,
|
||||
DEBUG_ONLY_THIS_PROCESS | CREATE_NO_WINDOW, nullptr, nullptr, &startup, &child))
|
||||
(debug ? DEBUG_ONLY_THIS_PROCESS : 0) | CREATE_SUSPENDED | CREATE_NO_WINDOW, nullptr, nullptr, &startup, &child))
|
||||
throw std::runtime_error("CreateProcess debugger error=" + std::to_string(GetLastError()));
|
||||
if (!AssignProcessToJobObject(job, child.hProcess)) {
|
||||
TerminateProcess(child.hProcess, 1); CloseHandle(job);
|
||||
throw std::runtime_error("AssignProcessToJobObject debugger failed");
|
||||
}
|
||||
log << "LAUNCH pid=" << child.dwProcessId << " attached_debugger=" << debug << std::endl;
|
||||
if (ResumeThread(child.hThread) == DWORD(-1)) {
|
||||
CloseHandle(job); throw std::runtime_error("Cannot resume diagnostic child");
|
||||
}
|
||||
if (!debug) {
|
||||
// Retain bounded process ownership and raw inherited output without
|
||||
// debugger stops on DLL/thread/exception/debug-string events.
|
||||
std::map<uintptr_t, Module> inventory;
|
||||
while (WaitForSingleObject(child.hProcess, 1000) == WAIT_TIMEOUT) {
|
||||
HANDLE snapshot = CreateToolhelp32Snapshot(TH32CS_SNAPMODULE | TH32CS_SNAPMODULE32, child.dwProcessId);
|
||||
if (snapshot == INVALID_HANDLE_VALUE) continue;
|
||||
MODULEENTRY32W module{sizeof(module)};
|
||||
if (Module32FirstW(snapshot, &module)) do {
|
||||
const auto base = reinterpret_cast<uintptr_t>(module.modBaseAddr);
|
||||
if (inventory.count(base)) continue;
|
||||
inventory[base] = {d4r::diag::utf8(module.szExePath), module.modBaseSize};
|
||||
log << "LOAD base=0x" << std::hex << base << " size=0x" << module.modBaseSize
|
||||
<< " path=" << inventory[base].path << '\n';
|
||||
} while (Module32NextW(snapshot, &module));
|
||||
CloseHandle(snapshot); log.flush();
|
||||
}
|
||||
DWORD result = 1;
|
||||
GetExitCodeProcess(child.hProcess, &result);
|
||||
log << "EXIT code=0x" << std::hex << result << std::endl;
|
||||
CloseHandle(child.hThread); CloseHandle(child.hProcess); CloseHandle(job);
|
||||
return static_cast<int>(result);
|
||||
}
|
||||
std::map<uintptr_t, Module> modules;
|
||||
bool initialBreakpoint = true, dumped = false; DWORD exitCode = 1;
|
||||
auto add_module = [&](void* base, HANDLE file) {
|
||||
@@ -73,7 +103,17 @@ int main(int argc, char** argv) {
|
||||
}
|
||||
};
|
||||
DEBUG_EVENT event{};
|
||||
uint64_t debugEvents = 0, debugStrings = 0, exceptions = 0;
|
||||
ULONGLONG lastSample = GetTickCount64();
|
||||
while (WaitForDebugEvent(&event, INFINITE)) {
|
||||
++debugEvents;
|
||||
if (event.dwDebugEventCode == OUTPUT_DEBUG_STRING_EVENT) ++debugStrings;
|
||||
if (event.dwDebugEventCode == EXCEPTION_DEBUG_EVENT) ++exceptions;
|
||||
const auto now = GetTickCount64();
|
||||
if (now - lastSample >= 5000) {
|
||||
log << "DEBUG_EVENTS events=" << std::dec << debugEvents << " debug_strings=" << debugStrings << " exceptions=" << exceptions << std::endl;
|
||||
lastSample = now;
|
||||
}
|
||||
DWORD disposition = DBG_CONTINUE;
|
||||
if (event.dwDebugEventCode == CREATE_PROCESS_DEBUG_EVENT) {
|
||||
add_module(event.u.CreateProcessInfo.lpBaseOfImage, event.u.CreateProcessInfo.hFile);
|
||||
|
||||
@@ -125,6 +125,8 @@ struct Args {
|
||||
std::string ngx_frontend = "d4r";
|
||||
std::string ngx_mode = "init";
|
||||
unsigned preset = 11;
|
||||
unsigned ngx_create_flags = 0;
|
||||
unsigned output_width = 512, output_height = 288;
|
||||
std::string fixture_dir;
|
||||
std::string output_dir;
|
||||
std::string kernel_name = "enc1";
|
||||
@@ -159,6 +161,22 @@ struct Args {
|
||||
ngx_frontend = value;
|
||||
}
|
||||
else if (key == "--ngx-mode") ngx_mode = value;
|
||||
else if (key == "--ngx-create-flags") {
|
||||
size_t end = 0;
|
||||
ngx_create_flags = static_cast<unsigned>(std::stoul(value, &end));
|
||||
if (end != value.size() || (ngx_create_flags != 0 && ngx_create_flags != 11))
|
||||
throw std::runtime_error("ngx-create-flags must be 0 or 11 (HDR, low-resolution MV, inverted depth)");
|
||||
}
|
||||
else if (key == "--ngx-output-resolution") {
|
||||
const auto separator = value.find('x');
|
||||
if (separator == std::string::npos) throw std::runtime_error("Output resolution must be WIDTHxHEIGHT");
|
||||
size_t wend = 0, hend = 0;
|
||||
output_width = static_cast<unsigned>(std::stoul(value.substr(0, separator), &wend));
|
||||
output_height = static_cast<unsigned>(std::stoul(value.substr(separator + 1), &hend));
|
||||
if (wend != separator || hend != value.size() - separator - 1 || output_width < 128 || output_width > 4096 ||
|
||||
output_height < 128 || output_height > 2160 || output_width % 2 || output_height % 2)
|
||||
throw std::runtime_error("Output resolution must be even and within 128x128..4096x2160");
|
||||
}
|
||||
else if (key == "--preset") {
|
||||
size_t end = 0;
|
||||
preset = static_cast<unsigned>(std::stoul(value, &end));
|
||||
|
||||
@@ -39,19 +39,27 @@ def metadata(directory):
|
||||
def report(directory, metadata_directory=None):
|
||||
kernel_metadata = metadata(metadata_directory)
|
||||
stages = collections.defaultdict(list)
|
||||
commands = collections.defaultdict(list)
|
||||
kernels = collections.defaultdict(list)
|
||||
for path in sorted(directory.glob('*.log')):
|
||||
text = path.read_text(encoding='utf-8-sig', errors='replace')
|
||||
for name, value in re.findall(r'D4R_STAGE name=(\S+) cpu_ms=([0-9.]+)', text):
|
||||
stages[name].append(float(value))
|
||||
for kind, fields in re.findall(r'(?m)^D4R_COMMAND_(RECORD|SUBMIT) ([^\r\n]+)', text):
|
||||
for name, value in re.findall(r'(\w+_ms)=([0-9.]+)', fields):
|
||||
number = float(value)
|
||||
if name != 'interval_ms' or number > 0:
|
||||
commands[kind.lower() + '.' + name].append(number)
|
||||
for line in text.splitlines():
|
||||
if not line.startswith('D4R_KERNEL_PROFILE '):
|
||||
continue
|
||||
fields = dict(re.findall(r'(\w+)=("[^"]+"|\S+)', line))
|
||||
kernels[(fields['kernel'].strip('"'), fields['backend'], fields['phase'])].append(fields)
|
||||
result = dict(cpuStages={name: stats(values) for name, values in stages.items()},
|
||||
commandStages={name: stats(values) for name, values in commands.items()},
|
||||
kernels=[], notes=[
|
||||
'CPU stage times include waits and host work; they are not isolated GPU timings.',
|
||||
'Command record intervals are between NGX recording calls on one thread, not Present/FPS.',
|
||||
'HIP-event profiling synchronizes every sampled launch and changes scheduling.',
|
||||
'Occupancy is an API prediction, not a measured hardware counter.',
|
||||
'Bandwidth and WMMA utilization require additional supported hardware tooling.'])
|
||||
@@ -83,6 +91,8 @@ def main():
|
||||
args.output.write_text(json.dumps(result, indent=2) + '\n', encoding='utf-8')
|
||||
for name, row in result['cpuStages'].items():
|
||||
print(f"CPU {name}: n={row['samples']} mean={row['mean']:.3f} ms median={row['median']:.3f} ms p95={row['p95']:.3f} ms")
|
||||
for name, row in result['commandStages'].items():
|
||||
print(f"COMMAND {name}: n={row['samples']} mean={row['mean']:.3f} ms median={row['median']:.3f} ms p95={row['p95']:.3f} ms")
|
||||
for row in result['kernels']:
|
||||
times = row['gpuMs']
|
||||
print(f"GPU {row['kernel']} {row['backend']} {row['phase']}: n={times['samples']} mean={times['mean']:.6f} ms total={times['total']:.3f} ms")
|
||||
|
||||
Reference in new issue
Block a user