diff --git a/docs/windows-game.md b/docs/windows-game.md index 0de82e3..ae1d383 100644 --- a/docs/windows-game.md +++ b/docs/windows-game.md @@ -3,8 +3,8 @@ Validated hardware: Windows 11 x64 / Radeon RX 9070 XT / gfx1201. Native K and M transformer kernels pass standalone D3D12 + OptiScaler checks. Silent Hill 2 K now renders real menu frames and M renders a saved gameplay level with -GPU NaN/Inf checks. Resolution coverage and performance are still being -validated. This is a local +GPU NaN/Inf checks. K also passes 3840x2160 output and M passes 1920x1080. +Further performance work is in progress. This is a local development package; no NVIDIA proprietary DLL is included. Build the shim/diagnostics, corrected ZLUDA, K/M native objects and patched @@ -29,8 +29,10 @@ Use `-Preset 13` for M. The validated manifest requires DLSS 310.9.1 with the recorded SHA256; other feature DLLs require regenerating and validating the native manifest. Select an upscaler supported by OptiScaler in the game. The script installs `dxgi.dll`, an explicit DLSS configuration and a private -`d4r` runtime directory, and captures stdout/stderr, an external debugger log, -crash dump, loaded DLL paths, driver information and OptiScaler logs. A local +`d4r` runtime directory, and captures stdout/stderr, exit code, loaded DLL paths, +driver information and OptiScaler logs. Ordinary runs use a process job without +an attached debugger. Add `-CaptureExceptions` to capture exception events and +minidumps when reproducing a crash. A local ZIP bundle is printed at exit. `-RunSeconds 120` bounds a diagnostic run; the normal default allows you to play until you close the game. @@ -46,7 +48,11 @@ the GPU before returning it to D3D12. Only two diagnostic counters (8 bytes) are read by the CPU; image data remains in VRAM. Validation failures are reported explicitly. Disable this diagnostic when measuring production speed. -`-ProfileStages` records CPU completion time for seven interop/NGX stages. +`-ProfileStages` records CPU completion time for seven interop/NGX stages, +command-list split/replay, submitted state publication and suffix completion. +The recording interval measures NGX calls on the recording thread, not Present. +`-ProfileKernels` enables synchronizing HIP-event kernel profiling and changes +frame scheduling; use it separately from a production FPS measurement. The runner drains stdout/stderr concurrently as raw bytes; the earlier line-oriented PowerShell collector imposed large per-frame delays. The optional ZLUDA kernel profiler is for diagnostics only. Measurements and reproduction @@ -58,6 +64,15 @@ kernel launches remain logged. `-VerboseRuntime` enables all API and frontend trace messages when investigating a failure. Numerical relaxation flags are cleared in the child process; FP16 rounding and subnormals stay enabled. +The optional K output-store optimization is built locally from your NVIDIA DLL +with `build-native-texture.ps1` and checked by `test-native-texture.ps1`; +commands and measurements are in [windows-performance.md](windows-performance.md). +Add `-LocalTextureKernels "C:\Users\Administrator\d4r\build\private-textures-gfx1201"` +to the game command to install it. The object/manifest hashes and DLSS identity +must match validation. These generated objects contain NVIDIA code, remain +private, and are excluded from the distributable package. Ordinary launches +inherit the game's resolution/window settings; the default argument is `-dx12`. + Original files are backed up before replacement. Restore with: ```powershell diff --git a/docs/windows-performance.md b/docs/windows-performance.md index 5582a72..104fbfc 100644 --- a/docs/windows-performance.md +++ b/docs/windows-performance.md @@ -73,5 +73,70 @@ Native FP8 is not the validated baseline and has no performance claim here. Replacing the exact per-MMA f16 rounding with f32 FP8 accumulation requires a new layer/reference validation; it is not justified by the observed CPU stalls. +Further game coverage (HIP kernel profiling disabled, GPU output checks enabled): +M at 1920x1080 completes 4833 frames, all finite. K at 3840x2160 completes +4685 frames, all finite. Neither has a backend failure, CPU image copy or an +aged-frame fallback. Their mean interop/NGX CPU stages total about 13.6 and +10.1 ms respectively; these are different scenes, not a K/M comparison. + +The no-debugger K/4K run completes 5774 frames, with 5774 finite output scans +and 63514 native transformer launches. The user reports 38-42 FPS. Additional +CPU measurements exclude replay/publication as the main cause of low FPS: + +| CPU measurement | Mean, ms | Median, ms | +| --- | ---: | ---: | +| Command split setup | 0.284 | 0.275 | +| State replay within split setup | 0.025 | 0.023 | +| Submitted texture state publication | 0.029 | 0.029 | +| Consumer suffix completion | 0.405 | 0.409 | +| NGX recording interval | 27.474 | 28.466 | + +Recording intervals are between NGX recording calls on the same thread, not +DXGI Present times. The interval includes rendering and application scheduling. +A second K/4K run with the attached debugger completes 3567 frames without +failure, with median recording interval 30.091 ms. The runs include different +menu/game states; this does not establish a precise debugger FPS speedup. +Its sparse debugger events also rule out a per-kernel OutputDebugString storm. +An earlier control that stayed on the startup screen completed zero DLSS frames +and is excluded. Normal game launches now omit the attached debugger; crash +diagnostics remain available with `-CaptureExceptions`. + +The 2961-frame 4K K profile identifies the translated output tail as its largest +kernel: 4.401670 ms mean GPU time, versus about 2.875 ms for all eleven native +transformer layers combined. Ported upstream's texture-store recipe to native +Windows. It retains accuracy mode, wave32, FP16 subnormals and f16 reference +accumulation; native FP8 remains disabled. + +Both output variants execute on every checked frame and match the previous +RGB output bit-exactly: eight consecutive 512x288 frames per variant, then +three consecutive 3840x2160 frames per variant. All RGBA components are finite. +Identical synthetic inputs and public NGX flags give these 4K event times +(three samples each; host initialization and readback are outside the event): + +| K output variant | Translated mean, ms | Native stores mean, ms | +| --- | ---: | ---: | +| LDR, regular depth, high-resolution MV | 2.907823 | 1.803233 | +| HDR, inverted depth, low-resolution MV | 2.978547 | 2.088347 | + +A subsequent game run executes 6221 frames with the native HDR output tail, +zero backend failures and mean tail GPU time 1.801918 ms. The earlier game +profile's 4.401670 ms is from a different scene/scheduling state; use the +identical-input table for the controlled comparison, not an inferred FPS +multiplier. This profiled game run did not enable the NaN scan. + +Reproduce the private texture build and full-frame comparisons: + +```powershell +.\scripts\windows\build-native-texture.ps1 -DlssDll "$PWD\nvngx_dlss.dll" +.\scripts\windows\build-native-texture.ps1 -DlssDll "$PWD\nvngx_dlss.dll" -Kernel hiluma_engine_output_depthreg_mvhi_ldr_max_v2_rel +.\scripts\windows\test-native-texture.ps1 -OutputResolution 3840x2160 -Iterations 3 +``` + +These objects contain code derived from the locally supplied NVIDIA PTX. They +stay in ignored local directories and are excluded from the public package. +The installer accepts them with `-LocalTextureKernels` only after validation, +checking the exact DLL identity, object and manifest hashes. M retains its +independently validated FP16-equivalent baseline. + References: [HIP events](https://rocmdocs.amd.com/projects/HIP/en/develop/doxygen/html/group___event.html), [HIP occupancy API](https://rocm.docs.amd.com/projects/HIP/en/latest/doxygen/html/group___occupancy.html). diff --git a/docs/windows-rdna4-port.md b/docs/windows-rdna4-port.md index 8dd02ee..ea7623c 100644 --- a/docs/windows-rdna4-port.md +++ b/docs/windows-rdna4-port.md @@ -9,6 +9,51 @@ validation coverage are recorded below. Performance remains in progress. ## Milestone and gates +Upstream refreshed and merged on 2026-10-01 through +[`f0d1a65`](https://github.com/countervolts/d4r/commit/f0d1a65e27aff6cbe7eeaa13227c80facb9056ab) +and artifact fix `65dfc9f`. Windows retains strict f16 rounding and the exact +M attention baseline. Upstream's optional denormal override now drops only +FP32 requirements, preserving FP16/FP64. All sixteen rebuilt native K/M +objects have the same SHA256 as the previously validated objects. All thirteen +ZLUDA patches apply to the pinned fresh base; all sixteen CTest gates pass. +The local ZLUDA source includes that denormal correction at `2378af7`; the +measured runtime remains the separately recorded `b0161a4` binary, with the +override unset. Package metadata records the binary build's actual commit. + +Additional production-style runs: M at 1920x1080 completes 4833 frames with +4833 finite-output GPU scans and no backend failures. K at 3840x2160 completes +4685 frames, 51535 native launches and 4685 finite-output GPU scans. Both use +current-frame VRAM interop, no CPU image copy and no previous-frame fallback. +The user confirms visible imagery at 4K but reports low FPS and about 25% GPU +usage. This percentage is not an isolated DLSS/WMMA hardware measurement. + +The game runner now launches without an attached debugger by default, while +retaining a kill-on-close process job, raw stdout/stderr, exit code, loaded +module inventory and frontend/driver logs. `-CaptureExceptions` enables the +original attached debugger and minidumps. A K/4K run without it completes +5774 frames / 63514 native launches / 5774 finite-output scans, zero failures. +The user reports 38-42 FPS. Recording/replay measurements: average split setup +0.284 ms, replay 0.025 ms, submitted texture-state publication 0.029 ms, +suffix completion 0.405 ms. These do not explain tens of milliseconds of +missing performance. NGX recording intervals average 27.474 ms, median +28.466 ms; they are not a DXGI Present measurement. The first debugger control +never left the startup screen and is explicitly not a valid FPS comparison. + +K output bottleneck: the 2961-frame 4K profile measures its translated output +tail at 4.401670 ms, larger than all eleven native transformer layers combined. +The Windows texture build preserves upstream Accuracy mode and creates private +NVIDIA-derived output objects. Both LDR and the game's HDR/inverted-depth/low-MV +variants pass strict full-frame comparisons: eight 512x288 frames each, then +three 3840x2160 frames each, max absolute/relative RGB error 0 and all RGBA +components finite. Controlled 4K tail GPU means: LDR 2.907823 -> 1.803233 ms; +HDR 2.978547 -> 2.088347 ms. The 6221-frame game profile executes the native tail +every frame, with zero backend failures and mean tail time 1.801918 ms. Scene +differences preclude an exact game FPS speedup claim. `-LocalTextureKernels` +requires matching source DLL, object and manifest hashes after validation; +public ZIPs exclude these objects. Install/restore with private overrides +restores sentinel DLL/config byte-exactly, without duplicate slash/backslash +manifest paths. + Performance diagnostics (2026-10-01): the separate ZLUDA `b0161a4` runtime passes 32 PTX launches in each context and all four native K/M frames with bit-exact RGB against their controls. All thirteen exported patches (0002 @@ -141,7 +186,7 @@ Do not integrate NGX until the integer PTX workload is stable on the real GPU. | M6: K layers, full transformer and image validation | PASS: 11 real-weight layers, 44 native launches, four D3D12 frames bit-exact against native CUDA, 81.85–92.91 dB versus translated K | | M7: M FP16-equivalent baseline and full transformer | PASS: all blocks of 40 native launches exactly match NumPy; targeted independent PTX regression matches; four D3D12 frames bit-exact against corrected translated M | | M8: standalone Windows D3D12 NGX harness | PASS for ordinary recorded EvaluateFeature and explicit boundaries: K/M, four frames each, no shim CPU image copies; both bit-exact against explicit boundary baselines | -| M9: OptiScaler integration, profiling, installation | Patched frontend K/M PASS; K game menu and M saved level PASS; profiling and final package pending | +| M9: OptiScaler integration, profiling, installation | Patched frontend and game K/M PASS, including K 4K and M 1080p finite output; native/translated GPU profiling and reversible install PASS; remaining FPS investigation and final archive in progress | ## Public NGX identity and OptiScaler gate (2026-10-01) diff --git a/scripts/windows/build-native-texture.ps1 b/scripts/windows/build-native-texture.ps1 new file mode 100644 index 0000000..d0d503f --- /dev/null +++ b/scripts/windows/build-native-texture.ps1 @@ -0,0 +1,15 @@ +[CmdletBinding()] +param( + [Parameter(Mandatory=$true)][string]$DlssDll, + [string]$Kernel = 'hiluma_engine_output_depthinv_mvlo_hdr_max_v2_rel', + [string]$HipRoot = 'C:\Program Files\AMD\ROCm\7.2', + [string]$ZludaRoot, + [string]$OutputDirectory +) +$ErrorActionPreference='Stop' +$repo=[IO.Path]::GetFullPath((Join-Path $PSScriptRoot '../..')) +if (!$ZludaRoot) { $ZludaRoot=Join-Path $repo 'dist/zluda-windows-profile' } +if (!$OutputDirectory) { $OutputDirectory=Join-Path $repo 'build/private-textures-gfx1201' } +& python (Join-Path $repo 'tools/windows/build_native_texture.py') --dlss-dll $DlssDll --hip-root $HipRoot ` + --zluda-root $ZludaRoot --kernel $Kernel --output-directory $OutputDirectory +if ($LASTEXITCODE) { throw "Native texture build failed ($LASTEXITCODE); inspect the output work directory." } diff --git a/scripts/windows/package-windows-game.ps1 b/scripts/windows/package-windows-game.ps1 index 937945a..7b87392 100644 --- a/scripts/windows/package-windows-game.ps1 +++ b/scripts/windows/package-windows-game.ps1 @@ -6,7 +6,7 @@ param( $ErrorActionPreference = 'Stop' $repo = [IO.Path]::GetFullPath((Join-Path $PSScriptRoot '../..')) $defaults = @{PackageRoot='dist/windows-rdna4-game'; DiagnosticRoot='dist/windows-rdna4-command-list'; - OptiScalerRoot='dist/optiscaler-windows-d4r'; ZludaRoot='dist/zluda-windows-f16-reference'; + OptiScalerRoot='dist/optiscaler-windows-d4r'; ZludaRoot='dist/zluda-windows-profile'; HipRoot='.tools/therock-10.2.0a20260929/_rocm_sdk_core'; KernelKRoot='build/native-k-gfx1201'; KernelMRoot='build/native-m-gfx1201'} foreach ($name in $defaults.Keys) { diff --git a/scripts/windows/test-native-texture.ps1 b/scripts/windows/test-native-texture.ps1 new file mode 100644 index 0000000..58e3e69 --- /dev/null +++ b/scripts/windows/test-native-texture.ps1 @@ -0,0 +1,52 @@ +[CmdletBinding()] +param([string]$TextureRoot, [string]$OutputDirectory, [int]$Iterations=8, [string]$OutputResolution='512x288') +$ErrorActionPreference='Stop' +$repo=[IO.Path]::GetFullPath((Join-Path $PSScriptRoot '../..')) +if (!$TextureRoot) { $TextureRoot=Join-Path $repo 'build/private-textures-gfx1201' } +if (!$OutputDirectory) { $OutputDirectory=Join-Path $repo 'test-results/k-native-texture-reference' } +if ($OutputResolution -notmatch '^([0-9]+)x([0-9]+)$') { throw 'OutputResolution must be WIDTHxHEIGHT.' } +$outputWidth=[int]$Matches[1]; $outputHeight=[int]$Matches[2] +$combined=Join-Path $OutputDirectory 'private-native' +New-Item -ItemType Directory -Force $combined | Out-Null +foreach ($directory in @((Join-Path $repo 'build/native-k-gfx1201'),$TextureRoot)) { + Get-ChildItem -LiteralPath $directory -Filter '*.hsaco' | Copy-Item -Destination $combined -Force +} +& python (Join-Path $repo 'kernels/tools/kernel_manifest.py') $combined (Join-Path $repo 'nvngx_dlss.dll') +if ($LASTEXITCODE) { throw 'Combined private manifest failed.' } +$settings=@{D4R_VALIDATE_OUTPUT='1'; D4R_PROFILE_STAGES='1'; D4R_ZLUDA_PROFILE='1'; D4R_QUIET_API='1'; + ZLUDA_CACHE_DIR=(Join-Path $repo 'build/zluda-cache-windows'); PYTHONPATH=(Join-Path $repo '.tools/python/vendor'); + D4R_ZLUDA_NATIVE_DIR=$null} +$old=@{} +try { + foreach ($key in $settings.Keys) { $old[$key]=[Environment]::GetEnvironmentVariable($key,'Process'); [Environment]::SetEnvironmentVariable($key,$settings[$key],'Process') } + foreach ($flags in @(0,11)) { + foreach ($mode in @('control','candidate')) { + $env:D4R_ZLUDA_NATIVE_DIR=if ($mode -eq 'control') { Join-Path $repo 'build/native-k-gfx1201' } else { $combined } + $arguments=@{RuntimeProfile='therock'; ZludaRoot=(Join-Path $repo 'dist/zluda-windows-profile'); + PackageRoot=(Join-Path $repo 'dist/windows-rdna4-command-list'); NgxCore=(Join-Path $repo '_nvngx.dll'); + DlssDll=(Join-Path $repo 'nvngx_dlss.dll'); NgxMode='d3d12'; NgxAbi='project-legacy'; Preset=11; + NgxCreateFlags=$flags; NgxOutputResolution=$OutputResolution; NgxOnly=$true; RequireNativeNetwork=$true; CommandListBackend=$true; + PixelProfile='depth-stencil'; BarrierMode='inherited-legacy'; CaptureExceptions=$true; EarlyIndirectProbe=$true; + Iterations=$Iterations; TimeoutSeconds=300; OptiScalerDll=(Join-Path $repo 'dist/optiscaler-windows-d4r/OptiScaler.dll'); + OutputDirectory=(Join-Path $OutputDirectory "$mode-$flags")} + & (Join-Path $PSScriptRoot 'test-windows-rdna4.ps1') @arguments + if ($LASTEXITCODE) { throw "Texture $mode flags=$flags workload failed." } + & python (Join-Path $repo 'tools/windows/profile_report.py') $arguments.OutputDirectory --output (Join-Path $arguments.OutputDirectory 'profile.json') + if ($LASTEXITCODE) { throw 'Texture profile report failed.' } + } + $control=Join-Path $OutputDirectory "control-$flags"; $candidate=Join-Path $OutputDirectory "candidate-$flags" + $kernel=if ($flags -eq 11) { 'hiluma_engine_output_depthinv_mvlo_hdr_max_v2_rel' } else { 'hiluma_engine_output_depthreg_mvhi_ldr_max_v2_rel' } + $stderr=Get-Content -LiteralPath (Join-Path $candidate 'd3d12-evaluate-preset-11.stderr.log') -Raw + $hits=[regex]::Matches($stderr, ('\[d4r-launch\] kernel="' + $kernel + '" backend=native')).Count + if ($hits -ne $Iterations) { throw "Texture override was not executed on every frame: $kernel hits=$hits" } + & python (Join-Path $repo 'tools/windows/frame_compare.py') --reference $control --actual $candidate --width $outputWidth --height $outputHeight --exact + if ($LASTEXITCODE) { throw "Texture flags=$flags full-frame RGB mismatch; preserve baseline." } + } + @{passed=$true; architecture='gfx1201'; framesPerVariant=$Iterations; outputResolution=$OutputResolution; createFlags=@(0,11); strictRgb=$true; + manifestSha256=(Get-FileHash -LiteralPath (Join-Path $TextureRoot 'd4r-kernels.txt') -Algorithm SHA256).Hash.ToLowerInvariant(); + privateNvidiaDerivedCode=$true; source=(Get-Content -LiteralPath (Join-Path $TextureRoot 'build-info.json') -Raw | ConvertFrom-Json)} | + ConvertTo-Json -Depth 12 | Set-Content -LiteralPath (Join-Path $TextureRoot 'validation.json') -Encoding UTF8 + Write-Host "PASS: both private K output variants executed and matched every RGB component. $OutputDirectory" +} finally { + foreach ($key in $old.Keys) { [Environment]::SetEnvironmentVariable($key,$old[$key],'Process') } +} diff --git a/scripts/windows/test-windows-rdna4.ps1 b/scripts/windows/test-windows-rdna4.ps1 index e8861a0..da2ed37 100644 --- a/scripts/windows/test-windows-rdna4.ps1 +++ b/scripts/windows/test-windows-rdna4.ps1 @@ -11,6 +11,8 @@ param( [ValidateSet('driver','sdk','project','project-legacy')][string]$NgxAbi = 'driver', [string]$OptiScalerDll, [ValidateSet(5, 11, 13)][int]$Preset = 11, + [ValidateSet(0,11)][int]$NgxCreateFlags = 0, + [string]$NgxOutputResolution = '512x288', [switch]$NgxOnly, [switch]$Trace, [switch]$RequireNativeNetwork, @@ -311,7 +313,8 @@ DisableSplash=true $summary.optiScaler = @{path=$OptiScalerDll; mode='standalone'; preset=$Preset} $ngxArguments += @('--ngx-frontend', 'optiscaler') } - $ngxArguments += @('--module', $localShim, '--pixel-profile', $PixelProfile, '--barrier-mode', $BarrierMode) + $ngxArguments += @('--module', $localShim, '--pixel-profile', $PixelProfile, '--barrier-mode', $BarrierMode, + '--ngx-create-flags', "$NgxCreateFlags", '--ngx-output-resolution', $NgxOutputResolution) if ($EarlyIndirectProbe) { if (!$CommandListBackend) { throw '-EarlyIndirectProbe requires -CommandListBackend' } $ngxArguments += @('--early-indirect', '1') @@ -366,7 +369,14 @@ DisableSplash=true [IO.Path]::GetFileName($resolvedRuntime) -notmatch '^d4r-ngx-[0-9a-f]{32}$') { throw 'Refusing to remove a path outside the private NGX temporary runtime.' } - Remove-Item -LiteralPath $resolvedRuntime -Recurse -Force + # A scanner can briefly hold the just-unloaded DLL open. Cleanup must + # not skip environment restoration or discard the completed GPU logs. + $cleanupError=$null + foreach ($attempt in 1..10) { + try { Remove-Item -LiteralPath $resolvedRuntime -Recurse -Force -ErrorAction Stop; $cleanupError=$null; break } + catch { $cleanupError=$_.Exception.Message; if ($attempt -lt 10) { Start-Sleep -Milliseconds 300 } } + } + if ($cleanupError) { $summary.cleanupError=$cleanupError; $exitStatus=1; Write-Warning "Private runtime cleanup failed: $cleanupError" } } $summary.passed = $exitStatus -eq 0 $summary | ConvertTo-Json -Depth 8 | Set-Content -LiteralPath (Join-Path $OutputDirectory 'summary.json') -Encoding UTF8 diff --git a/scripts/windows/windows-game.ps1 b/scripts/windows/windows-game.ps1 index ffff863..72ad141 100644 --- a/scripts/windows/windows-game.ps1 +++ b/scripts/windows/windows-game.ps1 @@ -7,10 +7,13 @@ param( [int]$RunSeconds = 0, [string]$DiagnosticResolution, [string]$CacheDirectory = $env:ZLUDA_CACHE_DIR, + [string]$LocalTextureKernels, [switch]$ValidateOutput, [switch]$ProfileStages, + [switch]$ProfileKernels, [switch]$VerboseRuntime, - [string[]]$GameArguments = @('-dx12','-windowed','-ResX=1280','-ResY=720'), + [switch]$CaptureExceptions, + [string[]]$GameArguments = @('-dx12'), [string]$OutputDirectory ) $ErrorActionPreference = 'Stop' @@ -56,13 +59,40 @@ foreach ($file in $metadata.files) { if ((Get-FileHash -LiteralPath (Join-Path $package $file.path) -Algorithm SHA256).Hash -ne $file.sha256) { throw "Package file changed; rebuild package: $($file.path)" } } if ((Get-FileHash -LiteralPath $DlssDll -Algorithm SHA256).Hash -ne $metadata.dlssSha256) { throw 'DLSS DLL does not match the validated 310.9.1 native manifest. Rebuild and validate native manifests for this DLL first.' } +$textureFiles=@(); $textureManifest=@() +if ($LocalTextureKernels) { + $LocalTextureKernels=(Get-Item -LiteralPath $LocalTextureKernels -ErrorAction Stop).FullName + $validation=Get-Content -LiteralPath (Join-Path $LocalTextureKernels 'validation.json') -Raw | ConvertFrom-Json + if (!$validation.passed -or !$validation.strictRgb -or $validation.architecture -ne 'gfx1201' -or + $validation.source.dlssSha256 -ne $metadata.dlssSha256 -or !$validation.source.accuracy) { + throw 'Local texture kernels must pass test-native-texture.ps1 with the exact packaged DLSS identity.' + } + foreach ($object in $validation.source.objects.PSObject.Properties) { + if ($object.Name -notin @('hiluma_engine_output_depthinv_mvlo_hdr_max_v2_rel.hsaco','hiluma_engine_output_depthreg_mvhi_ldr_max_v2_rel.hsaco')) { + throw "Unvalidated local texture variant: $($object.Name)" + } + $path=Join-Path $LocalTextureKernels $object.Name + if ((Get-FileHash -LiteralPath $path -Algorithm SHA256).Hash.ToLowerInvariant() -ne $object.Value) { throw 'Local texture object changed after validation.' } + $textureFiles+=Get-Item -LiteralPath $path + } + if ($textureFiles.Count -ne 2) { throw 'Expected both validated local K output variants.' } + # Use the manifest covered by validation, rather than a mutable extra file. + foreach ($line in Get-Content -LiteralPath (Join-Path $LocalTextureKernels 'd4r-kernels.txt')) { + if (!$line.Trim() -or $line.StartsWith('#')) { continue } + if ($line -notmatch '^([A-Za-z0-9_]+) ([0-9a-f]{16})$' -or ($Matches[1]+'.hsaco') -notin @($textureFiles.Name)) { throw 'Invalid local texture manifest.' } + $textureManifest+=$line + } + if ($textureManifest.Count -ne 2) { throw 'Expected two local texture manifest identities.' } + if ((Get-FileHash -LiteralPath (Join-Path $LocalTextureKernels 'd4r-kernels.txt') -Algorithm SHA256).Hash.ToLowerInvariant() -ne $validation.manifestSha256) { throw 'Local texture manifest changed after validation.' } +} $manifest = if (Test-Path -LiteralPath $manifestPath) { Get-Content -LiteralPath $manifestPath -Raw | ConvertFrom-Json } else { [pscustomobject]@{game=$GameExe; files=@()} } if ($manifest.game -ne $GameExe) { throw 'Backup manifest belongs to another executable.' } if (!(Test-Path -LiteralPath $manifestPath) -and (Test-Path -LiteralPath (GamePath 'd4r'))) { throw 'An existing d4r directory has no installation manifest; preserved.' } New-Item -ItemType Directory -Force $backup | Out-Null function InstallFile([string]$source, [string]$relative) { + $relative=$relative.Replace('/','\') $path = GamePath $relative - $entry = @($manifest.files | Where-Object path -eq $relative) | Select-Object -First 1 + $entry = @($manifest.files | Where-Object { $_.path.Replace('/','\') -eq $relative }) | Select-Object -First 1 if ($entry) { if ((Test-Path -LiteralPath $path) -and (Get-FileHash -LiteralPath $path -Algorithm SHA256).Hash -ne $entry.installedHash) { throw "Installed file changed; preserved: $path" } } else { @@ -82,6 +112,13 @@ function InstallFile([string]$source, [string]$relative) { } InstallFile (Join-Path $package 'OptiScaler.dll') 'dxgi.dll' foreach ($file in Get-ChildItem -LiteralPath (Join-Path $package 'd4r') -Recurse -File) { InstallFile $file.FullName $file.FullName.Substring($package.Length+1) } +if ($LocalTextureKernels) { + foreach ($file in $textureFiles) { InstallFile $file.FullName ('d4r/native/' + $file.Name) } + $combinedManifest=Join-Path $backup 'generated-native-manifest.txt' + @(Get-Content -LiteralPath (Join-Path $package 'd4r/native/d4r-kernels.txt')) + $textureManifest | + Set-Content -LiteralPath $combinedManifest -Encoding ASCII + InstallFile $combinedManifest 'd4r/native/d4r-kernels.txt' +} # Private local test copies only; these are never added to the public package. InstallFile $NgxCore 'd4r/vendor/_nvngx.dll' InstallFile $DlssDll 'd4r/vendor/nvngx_dlss.dll' @@ -136,13 +173,14 @@ $settings = @{ D4R_ZLUDA_IGNORE_DENORMAL=$null; D4R_ZLUDA_FAST_MATH=$null; D4R_ZLUDA_WMMA_F32ACC=$null; D4R_ZLUDA_WAVE64=$null; D4R_QUIET_API=$(if ($VerboseRuntime) { $null } else { '1' }); + D4R_VALIDATE_OUTPUT=$(if ($ValidateOutput) { '1' } else { $null }); + D4R_PROFILE_STAGES=$(if ($ProfileStages) { '1' } else { $null }); + D4R_ZLUDA_PROFILE=$(if ($ProfileKernels) { '1' } else { $null }); D4R_DIAG_DIR=$OutputDirectory; ZLUDA_LOG_DIR=(Join-Path $OutputDirectory 'zluda-trace'); ZLUDA_CACHE_DIR=$CacheDirectory; PATH=((GamePath 'd4r/hip/bin') + ';' + (GamePath 'd4r/zluda') + ';' + $env:PATH) } $old = @{}; $process = $null; $originalSettings = @() $outFile = $null; $errFile = $null; $outTask = $null; $errTask = $null -if ($ValidateOutput) { $settings['D4R_VALIDATE_OUTPUT']='1' } -if ($ProfileStages) { $settings['D4R_PROFILE_STAGES']='1' } try { if ($DiagnosticResolution) { if ([IO.Path]::GetFileName($GameExe) -ne 'SHProto-Win64-Shipping.exe' -or $DiagnosticResolution -notmatch '^([0-9]{3,4})x([0-9]{3,4})$') { throw 'DiagnosticResolution currently supports Silent Hill 2 only; use e.g. 1280x720.' } @@ -173,10 +211,12 @@ try { Write-Host "Temporary diagnostic resolution: $DiagnosticResolution; original settings restore at exit." } foreach ($key in $settings.Keys) { $old[$key] = [Environment]::GetEnvironmentVariable($key,'Process'); [Environment]::SetEnvironmentVariable($key,$settings[$key],'Process') } - @{preset=$Preset; game=$GameExe; package=$metadata; driver=@(Get-CimInstance Win32_VideoController | Select-Object Name,DriverVersion,DriverDate); arguments=$GameArguments} | + @{preset=$Preset; game=$GameExe; package=$metadata; attachedDebugger=[bool]$CaptureExceptions; driver=@(Get-CimInstance Win32_VideoController | Select-Object Name,DriverVersion,DriverDate); arguments=$GameArguments} | ConvertTo-Json -Depth 12 | Set-Content -LiteralPath (Join-Path $OutputDirectory 'environment.json') -Encoding UTF8 function Quote([string]$value) { if ($value.Contains('"')) { throw 'A command argument contains an unsupported quote.' }; return '"' + $value + '"' } - $arguments = @('--output-directory', (Quote $OutputDirectory), '--', (Quote $GameExe)) + @($GameArguments | ForEach-Object { Quote $_ }) + $arguments = @('--output-directory', (Quote $OutputDirectory)) + if (!$CaptureExceptions) { $arguments += '--no-debug' } + $arguments += @('--', (Quote $GameExe)) + @($GameArguments | ForEach-Object { Quote $_ }) # Drain raw bytes concurrently. PowerShell's line-oriented redirection can # throttle a verbose game and distort the very timings we are collecting. $info = [Diagnostics.ProcessStartInfo]::new() @@ -186,7 +226,7 @@ try { $info.UseShellExecute = $false; $info.CreateNoWindow = $true $info.RedirectStandardOutput = $true; $info.RedirectStandardError = $true $process = [Diagnostics.Process]::new(); $process.StartInfo = $info - if (!$process.Start()) { throw 'Game debugger launch failed.' } + if (!$process.Start()) { throw 'Game diagnostic launch failed.' } $outFile = [IO.File]::Open((Join-Path $OutputDirectory 'd4r.stdout.log'), [IO.FileMode]::Create, [IO.FileAccess]::Write, [IO.FileShare]::Read) $errFile = [IO.File]::Open((Join-Path $OutputDirectory 'd4r.stderr.log'), [IO.FileMode]::Create, [IO.FileAccess]::Write, [IO.FileShare]::Read) $outTask = $process.StandardOutput.BaseStream.CopyToAsync($outFile) diff --git a/tools/windows/build_native_texture.py b/tools/windows/build_native_texture.py new file mode 100644 index 0000000..d5d7c1a --- /dev/null +++ b/tools/windows/build_native_texture.py @@ -0,0 +1,90 @@ +"""Build a private gfx1201 texture override from the user's local DLSS DLL. + +The resulting PTX/code object contains NVIDIA code and must remain local. +No Wine, shell script, GPU, CUDA SDK or system installation is required. +""" +import argparse +import hashlib +import json +import os +import pathlib +import re +import shutil +import subprocess +import sys + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--dlss-dll', type=pathlib.Path, required=True) + parser.add_argument('--hip-root', type=pathlib.Path, required=True) + parser.add_argument('--zluda-root', type=pathlib.Path, required=True) + parser.add_argument('--kernel', required=True) + parser.add_argument('--output-directory', type=pathlib.Path, required=True) + args = parser.parse_args() + if not re.fullmatch(r'hiluma_engine_output_depth(?:inv|reg)_mv(?:hi|lo)_(?:hdr|ldr)(?:_max)?_v2_rel', args.kernel): + parser.error('this initial validated build recipe supports K v2 output variants only') + repo = pathlib.Path(__file__).resolve().parents[2] + output = args.output_directory.resolve() + output.mkdir(parents=True, exist_ok=True) + work = output / ('work-' + args.kernel) + work.mkdir(exist_ok=True) + hip = args.hip_root.resolve() + zluda = args.zluda_root.resolve() + tools = {name: hip / 'bin' / (name + '.exe') for name in ('clang++', 'llvm-dis', 'llvm-as')} + emitter = zluda / 'd4r_emit.exe' + for path in [args.dlss_dll, emitter, *tools.values()]: + if not path.is_file(): + parser.error(f'missing dependency: {path}') + env = os.environ.copy() + env.update(D4R_PREFER_ACCURACY='1', D4R_ZLUDA_WMMA='1', D4R_ZLUDA_WMMA_FP8='1', + D4R_ZLUDA_WMMA_FP8_NATIVE='0', D4R_ZLUDA_WMMA_F16_REFERENCE='1', + D4R_ZLUDA_IMPLICIT_MAX_BLOCK='256') + for name in ('D4R_ZLUDA_IGNORE_DENORMAL', 'D4R_ZLUDA_FAST_MATH', 'D4R_ZLUDA_WMMA_F32ACC', + 'D4R_ZLUDA_WAVE64', 'D4R_ZLUDA_PROFILE'): + env.pop(name, None) + env['PATH'] = str(zluda) + os.pathsep + str(hip / 'bin') + os.pathsep + env['PATH'] + + def run(label, command): + with (work / (label + '.log')).open('wb') as log: + result = subprocess.run([str(arg) for arg in command], env=env, stdout=log, stderr=subprocess.STDOUT) + if result.returncode: + print((work / (label + '.log')).read_text(errors='replace')[-8000:], file=sys.stderr) + raise RuntimeError(f'{label} failed ({result.returncode}); see {work}') + + ptx = work / 'ptx' + run('extract', [sys.executable, repo / 'kernels/tools/extract_dlss_ptx.py', args.dlss_dll, ptx]) + env['D4R_DLSS_PTX_DIR'] = str(ptx) + edited = work / (args.kernel + '.ptx') + run('make-ptx', [sys.executable, repo / 'kernels/tex/make_ptx.py', args.kernel, edited]) + raw, ir, extra = work / 'raw.bc', work / 'raw.ll', work / 'extra.bc' + run('compile-helper', [tools['clang++'], '-x', 'hip', '-std=c++20', '-nogpuinc', '-nogpulib', '-O3', + '-mno-wavefrontsize64', '--offload-device-only', '--offload-arch=gfx1201', + '-fgpu-rdc', '-emit-llvm', '-c', '-Xclang', '-fdenormal-fp-math=dynamic', + '-DD4R_ACCURACY', '-DD4R_KERNEL_NAME=' + args.kernel, + '-o', raw, repo / 'kernels/tex/sust_only.hip']) + run('disassemble', [tools['llvm-dis'], raw, '-o', ir]) + lines = [] + for line in ir.read_text().splitlines(): + if re.search(r'@llvm.used|wchar_size|llvm.module.flags|__hip_cuid', line): + continue + line = line.replace('optnone', '').replace('"target-cpu"="gfx1201"', '') + lines.append(re.sub(r'"target-features"="[^"]+"', '', line)) + ir.write_text('\n'.join(lines) + '\n') + run('assemble', [tools['llvm-as'], ir, '-o', extra]) + env['D4R_ZLUDA_EXTRA_BC'] = str(extra) + run('emit', [emitter, edited, work / 'emitted', 'gfx1201']) + shutil.copyfile(work / 'emitted/module.hsaco', output / (args.kernel + '.hsaco')) + run('manifest', [sys.executable, repo / 'kernels/tools/kernel_manifest.py', output, args.dlss_dll]) + metadata = dict(architecture='gfx1201', accuracy=True, nativeFP8=False, + privateNvidiaDerivedCode=True, validation='not yet validated; do not install before comparison', + dlssSha256=hashlib.sha256(args.dlss_dll.read_bytes()).hexdigest(), + zludaBuild=json.loads((zluda / 'build-info.json').read_text(encoding='utf-8-sig')), + compiler=subprocess.check_output([str(tools['clang++']), '--version'], env=env).decode(errors='replace'), + objects={p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in output.glob('*.hsaco')}) + (output / 'build-info.json').write_text(json.dumps(metadata, indent=2) + '\n') + print(f'Built PRIVATE gfx1201 output override: {output}; validation required before use') + + +if __name__ == '__main__': + main() diff --git a/tools/windows/d3d12_command_hooks.cpp b/tools/windows/d3d12_command_hooks.cpp index d6027e3..cfce542 100644 --- a/tools/windows/d3d12_command_hooks.cpp +++ b/tools/windows/d3d12_command_hooks.cpp @@ -6,6 +6,7 @@ #include #include #include +#include namespace d4r::win::commands { namespace { @@ -13,6 +14,9 @@ thread_local bool internal = false; std::mutex map_mutex, installation_mutex, error_mutex; std::string last_error; bool installed = false; +using Clock = std::chrono::steady_clock; +bool profile_commands() { static const bool enabled = std::getenv("D4R_PROFILE_STAGES") != nullptr; return enabled; } +double milliseconds(Clock::time_point start) { return std::chrono::duration(Clock::now() - start).count(); } struct Event { UINT metadata; std::vector data; }; using State = std::function; struct TrackedResource { @@ -385,6 +389,8 @@ void complete(ID3D12CommandQueue* queue) { } void STDMETHODCALLTYPE hook_execute(ID3D12CommandQueue* queue, UINT count, ID3D12CommandList* const* lists) { if (internal) { original_execute(queue, count, lists); return; } + const auto submitStart = Clock::now(); + double publishMs = 0, boundaryMs = 0, drainMs = 0; std::vector> owners; std::vector pending; std::vector pending_states; @@ -394,7 +400,9 @@ void STDMETHODCALLTYPE hook_execute(ID3D12CommandQueue* queue, UINT count, ID3D1 original_execute(queue, UINT(pending.size()), pending.data()); // Publish in the actual submitted order, never while recording. // The callback's external fence waits for these producers on GPU. + const auto publishStart = Clock::now(); for (const auto& states : pending_states) publish_states(states); + publishMs += milliseconds(publishStart); pending.clear(); pending_states.clear(); } }; @@ -416,13 +424,19 @@ void STDMETHODCALLTYPE hook_execute(ID3D12CommandQueue* queue, UINT count, ID3D1 owners.push_back(recording); boundaries = true; for (const auto& segment : recording->segments) { pending.push_back(segment.list); pending_states.push_back(segment.resource_states); flush(); + const auto boundaryStart = Clock::now(); segment.boundary(queue); + boundaryMs += milliseconds(boundaryStart); } pending.push_back(recording->current.Get()); pending_states.push_back(recording->resource_states); } flush(); - if (boundaries) complete(queue); + if (boundaries) { + const auto drainStart = Clock::now(); complete(queue); drainMs = milliseconds(drainStart); + if (profile_commands()) std::printf("D4R_COMMAND_SUBMIT total_ms=%.3f boundary_ms=%.3f publish_ms=%.3f suffix_drain_ms=%.3f\n", + milliseconds(submitStart), boundaryMs, publishMs, drainMs); + } } catch (const std::exception& failure) { { std::lock_guard lock(error_mutex); last_error = failure.what(); } std::fprintf(stderr, "D4R_D3D12_SUBMIT_FAILURE %s\n", failure.what()); @@ -510,6 +524,7 @@ void install(ID3D12Device* device) { } } void record_boundary(ID3D12GraphicsCommandList* list, Boundary callback) { + const auto recordStart = Clock::now(); if (!installed || !list || !callback) throw std::runtime_error("D3D12 command backend is not initialized"); if (!error().empty()) throw std::runtime_error(error()); if (list->GetType() != D3D12_COMMAND_LIST_TYPE_DIRECT) throw std::runtime_error("DLSS requires a direct command list"); @@ -527,7 +542,9 @@ void record_boundary(ID3D12GraphicsCommandList* list, Boundary callback) { ComPtr nextAllocator; ComPtr next; dx(device->CreateCommandAllocator(D3D12_COMMAND_LIST_TYPE_DIRECT, IID_PPV_ARGS(nextAllocator.GetAddressOf())), "Suffix allocator"); dx(device->CreateCommandList(0, D3D12_COMMAND_LIST_TYPE_DIRECT, nextAllocator.Get(), nullptr, IID_PPV_ARGS(next.GetAddressOf())), "Suffix command list"); + const auto replayStart = Clock::now(); for (auto& state : recording.state) state(next.Get()); + const auto replayMs = milliseconds(replayStart); recording.segments.reserve(recording.segments.size() + 1); for (const auto& event : recording.events) original_BeginEvent(next.Get(), event.metadata, event.data.data(), UINT(event.data.size())); for (size_t n = 0; n < recording.events.size(); ++n) original_EndEvent(segment.list); @@ -537,5 +554,14 @@ void record_boundary(ID3D12GraphicsCommandList* list, Boundary callback) { recording.current_allocator = std::move(nextAllocator); recording.current = std::move(next); std::printf("D4R_D3D12_RECORD boundary=%zu state_calls=%zu frame_age=0\n", recording.segments.size(), recording.state.size()); + if (profile_commands()) { + // This is the interval between NGX recording calls on this thread, + // not a Present/FPS measurement or an extra queued frame. + thread_local Clock::time_point previous{}; + const double interval = previous == Clock::time_point{} ? 0 : std::chrono::duration(recordStart - previous).count(); + previous = recordStart; + std::printf("D4R_COMMAND_RECORD total_ms=%.3f replay_ms=%.3f state_calls=%zu interval_ms=%.3f\n", + milliseconds(recordStart), replayMs, recording.state.size(), interval); + } } } diff --git a/tools/windows/d3d12_dlss_probe.cpp b/tools/windows/d3d12_dlss_probe.cpp index 4962693..fd2cb23 100644 --- a/tools/windows/d3d12_dlss_probe.cpp +++ b/tools/windows/d3d12_dlss_probe.cpp @@ -73,7 +73,9 @@ int main(int argc, char** argv) { ComPtr list; dx(device->CreateCommandAllocator(D3D12_COMMAND_LIST_TYPE_DIRECT, IID_PPV_ARGS(allocator.GetAddressOf())), "CreateCommandAllocator"); dx(device->CreateCommandList(0, D3D12_COMMAND_LIST_TYPE_DIRECT, allocator.Get(), nullptr, IID_PPV_ARGS(list.GetAddressOf())), "CreateCommandList"); - constexpr unsigned width = 256, height = 144, outWidth = 512, outHeight = 288; + const unsigned outWidth = args.output_width, outHeight = args.output_height; + const unsigned width = outWidth / 2, height = outHeight / 2; + std::printf("D3D12_NGX_DIMENSIONS input=%ux%u output=%ux%u\n", width, height, outWidth, outHeight); struct Texture { ComPtr image, upload; D3D12_PLACED_SUBRESOURCE_FOOTPRINT footprint{}; UINT64 bytes; } textures[5]; for (unsigned i = 0; i < 5; ++i) { auto& texture = textures[i]; @@ -230,7 +232,8 @@ int main(int argc, char** argv) { for (const auto& pair : {std::make_pair("Width", width), std::make_pair("Height", height), std::make_pair("OutWidth", outWidth), std::make_pair("OutHeight", outHeight), std::make_pair("CreationNodeMask", 1u), std::make_pair("VisibilityNodeMask", 1u)}) d4r_ngx_set_uint(parameters, pair.first, pair.second); - d4r_ngx_set_int(parameters, "PerfQualityValue", 2); d4r_ngx_set_int(parameters, "DLSS.Feature.Create.Flags", 0); + d4r_ngx_set_int(parameters, "PerfQualityValue", 2); d4r_ngx_set_int(parameters, "DLSS.Feature.Create.Flags", args.ngx_create_flags); + std::printf("D3D12_NGX_CREATE_FLAGS value=%u\n", args.ngx_create_flags); for (const char* name : {"DLSS.Hint.Render.Preset.DLAA", "DLSS.Hint.Render.Preset.Quality", "DLSS.Hint.Render.Preset.Balanced", "DLSS.Hint.Render.Preset.Performance", "DLSS.Hint.Render.Preset.UltraPerformance", "DLSS.Hint.Render.Preset.UltraQuality"}) d4r_ngx_set_uint(parameters, name, args.preset); void* handle = nullptr; diff --git a/tools/windows/debug_launcher.cpp b/tools/windows/debug_launcher.cpp index a9fed60..f7fa337 100644 --- a/tools/windows/debug_launcher.cpp +++ b/tools/windows/debug_launcher.cpp @@ -2,6 +2,7 @@ #include "diagnostic.h" #include #include +#include namespace { std::wstring quote(const std::wstring& value) { @@ -31,14 +32,16 @@ uint32_t module_size(HANDLE process, uintptr_t base) { } int main(int argc, char** argv) { try { - if (argc < 5 || std::strcmp(argv[1], "--output-directory") || std::strcmp(argv[3], "--")) - throw std::runtime_error("Usage: d4r_debug_launcher --output-directory PATH -- EXE [arguments]"); + const bool debug = argc < 4 || std::strcmp(argv[3], "--no-debug"); + const int separator = debug ? 3 : 4; + if (argc <= separator + 1 || std::strcmp(argv[1], "--output-directory") || std::strcmp(argv[separator], "--")) + throw std::runtime_error("Usage: d4r_debug_launcher --output-directory PATH [--no-debug] -- EXE [arguments]"); const std::filesystem::path directory(d4r::diag::wide(argv[2])); std::filesystem::create_directories(directory); std::ofstream log(directory / L"debugger.log", std::ios::app); - const auto executable = d4r::diag::wide(argv[4]); + const auto executable = d4r::diag::wide(argv[separator + 1]); std::wstring command; - for (int i = 4; i < argc; ++i) { if (i > 4) command += L' '; command += quote(d4r::diag::wide(argv[i])); } + for (int i = separator + 1; i < argc; ++i) { if (i > separator + 1) command += L' '; command += quote(d4r::diag::wide(argv[i])); } STARTUPINFOW startup{sizeof(startup)}; PROCESS_INFORMATION child{}; startup.dwFlags = STARTF_USESTDHANDLES; startup.hStdInput = GetStdHandle(STD_INPUT_HANDLE); @@ -50,12 +53,39 @@ int main(int argc, char** argv) { if (!job || !SetInformationJobObject(job, JobObjectExtendedLimitInformation, &limits, sizeof(limits))) throw std::runtime_error("Cannot create diagnostic process lifetime job"); if (!CreateProcessW(executable.c_str(), command.data(), nullptr, nullptr, TRUE, - DEBUG_ONLY_THIS_PROCESS | CREATE_NO_WINDOW, nullptr, nullptr, &startup, &child)) + (debug ? DEBUG_ONLY_THIS_PROCESS : 0) | CREATE_SUSPENDED | CREATE_NO_WINDOW, nullptr, nullptr, &startup, &child)) throw std::runtime_error("CreateProcess debugger error=" + std::to_string(GetLastError())); if (!AssignProcessToJobObject(job, child.hProcess)) { TerminateProcess(child.hProcess, 1); CloseHandle(job); throw std::runtime_error("AssignProcessToJobObject debugger failed"); } + log << "LAUNCH pid=" << child.dwProcessId << " attached_debugger=" << debug << std::endl; + if (ResumeThread(child.hThread) == DWORD(-1)) { + CloseHandle(job); throw std::runtime_error("Cannot resume diagnostic child"); + } + if (!debug) { + // Retain bounded process ownership and raw inherited output without + // debugger stops on DLL/thread/exception/debug-string events. + std::map inventory; + while (WaitForSingleObject(child.hProcess, 1000) == WAIT_TIMEOUT) { + HANDLE snapshot = CreateToolhelp32Snapshot(TH32CS_SNAPMODULE | TH32CS_SNAPMODULE32, child.dwProcessId); + if (snapshot == INVALID_HANDLE_VALUE) continue; + MODULEENTRY32W module{sizeof(module)}; + if (Module32FirstW(snapshot, &module)) do { + const auto base = reinterpret_cast(module.modBaseAddr); + if (inventory.count(base)) continue; + inventory[base] = {d4r::diag::utf8(module.szExePath), module.modBaseSize}; + log << "LOAD base=0x" << std::hex << base << " size=0x" << module.modBaseSize + << " path=" << inventory[base].path << '\n'; + } while (Module32NextW(snapshot, &module)); + CloseHandle(snapshot); log.flush(); + } + DWORD result = 1; + GetExitCodeProcess(child.hProcess, &result); + log << "EXIT code=0x" << std::hex << result << std::endl; + CloseHandle(child.hThread); CloseHandle(child.hProcess); CloseHandle(job); + return static_cast(result); + } std::map modules; bool initialBreakpoint = true, dumped = false; DWORD exitCode = 1; auto add_module = [&](void* base, HANDLE file) { @@ -73,7 +103,17 @@ int main(int argc, char** argv) { } }; DEBUG_EVENT event{}; + uint64_t debugEvents = 0, debugStrings = 0, exceptions = 0; + ULONGLONG lastSample = GetTickCount64(); while (WaitForDebugEvent(&event, INFINITE)) { + ++debugEvents; + if (event.dwDebugEventCode == OUTPUT_DEBUG_STRING_EVENT) ++debugStrings; + if (event.dwDebugEventCode == EXCEPTION_DEBUG_EVENT) ++exceptions; + const auto now = GetTickCount64(); + if (now - lastSample >= 5000) { + log << "DEBUG_EVENTS events=" << std::dec << debugEvents << " debug_strings=" << debugStrings << " exceptions=" << exceptions << std::endl; + lastSample = now; + } DWORD disposition = DBG_CONTINUE; if (event.dwDebugEventCode == CREATE_PROCESS_DEBUG_EVENT) { add_module(event.u.CreateProcessInfo.lpBaseOfImage, event.u.CreateProcessInfo.hFile); diff --git a/tools/windows/diagnostic.h b/tools/windows/diagnostic.h index e905d14..6161a9f 100644 --- a/tools/windows/diagnostic.h +++ b/tools/windows/diagnostic.h @@ -125,6 +125,8 @@ struct Args { std::string ngx_frontend = "d4r"; std::string ngx_mode = "init"; unsigned preset = 11; + unsigned ngx_create_flags = 0; + unsigned output_width = 512, output_height = 288; std::string fixture_dir; std::string output_dir; std::string kernel_name = "enc1"; @@ -159,6 +161,22 @@ struct Args { ngx_frontend = value; } else if (key == "--ngx-mode") ngx_mode = value; + else if (key == "--ngx-create-flags") { + size_t end = 0; + ngx_create_flags = static_cast(std::stoul(value, &end)); + if (end != value.size() || (ngx_create_flags != 0 && ngx_create_flags != 11)) + throw std::runtime_error("ngx-create-flags must be 0 or 11 (HDR, low-resolution MV, inverted depth)"); + } + else if (key == "--ngx-output-resolution") { + const auto separator = value.find('x'); + if (separator == std::string::npos) throw std::runtime_error("Output resolution must be WIDTHxHEIGHT"); + size_t wend = 0, hend = 0; + output_width = static_cast(std::stoul(value.substr(0, separator), &wend)); + output_height = static_cast(std::stoul(value.substr(separator + 1), &hend)); + if (wend != separator || hend != value.size() - separator - 1 || output_width < 128 || output_width > 4096 || + output_height < 128 || output_height > 2160 || output_width % 2 || output_height % 2) + throw std::runtime_error("Output resolution must be even and within 128x128..4096x2160"); + } else if (key == "--preset") { size_t end = 0; preset = static_cast(std::stoul(value, &end)); diff --git a/tools/windows/profile_report.py b/tools/windows/profile_report.py index a8e42dd..1464396 100644 --- a/tools/windows/profile_report.py +++ b/tools/windows/profile_report.py @@ -39,19 +39,27 @@ def metadata(directory): def report(directory, metadata_directory=None): kernel_metadata = metadata(metadata_directory) stages = collections.defaultdict(list) + commands = collections.defaultdict(list) kernels = collections.defaultdict(list) for path in sorted(directory.glob('*.log')): text = path.read_text(encoding='utf-8-sig', errors='replace') for name, value in re.findall(r'D4R_STAGE name=(\S+) cpu_ms=([0-9.]+)', text): stages[name].append(float(value)) + for kind, fields in re.findall(r'(?m)^D4R_COMMAND_(RECORD|SUBMIT) ([^\r\n]+)', text): + for name, value in re.findall(r'(\w+_ms)=([0-9.]+)', fields): + number = float(value) + if name != 'interval_ms' or number > 0: + commands[kind.lower() + '.' + name].append(number) for line in text.splitlines(): if not line.startswith('D4R_KERNEL_PROFILE '): continue fields = dict(re.findall(r'(\w+)=("[^"]+"|\S+)', line)) kernels[(fields['kernel'].strip('"'), fields['backend'], fields['phase'])].append(fields) result = dict(cpuStages={name: stats(values) for name, values in stages.items()}, + commandStages={name: stats(values) for name, values in commands.items()}, kernels=[], notes=[ 'CPU stage times include waits and host work; they are not isolated GPU timings.', + 'Command record intervals are between NGX recording calls on one thread, not Present/FPS.', 'HIP-event profiling synchronizes every sampled launch and changes scheduling.', 'Occupancy is an API prediction, not a measured hardware counter.', 'Bandwidth and WMMA utilization require additional supported hardware tooling.']) @@ -83,6 +91,8 @@ def main(): args.output.write_text(json.dumps(result, indent=2) + '\n', encoding='utf-8') for name, row in result['cpuStages'].items(): print(f"CPU {name}: n={row['samples']} mean={row['mean']:.3f} ms median={row['median']:.3f} ms p95={row['p95']:.3f} ms") + for name, row in result['commandStages'].items(): + print(f"COMMAND {name}: n={row['samples']} mean={row['mean']:.3f} ms median={row['median']:.3f} ms p95={row['p95']:.3f} ms") for row in result['kernels']: times = row['gpuMs'] print(f"GPU {row['kernel']} {row['backend']} {row['phase']}: n={times['samples']} mean={times['mean']:.6f} ms total={times['total']:.3f} ms")