engine: presets E/F AutoExposure, the game's exposure fields and NGX's jitter order

NGX's CNN takes this frame's exposure scale and pre-exposure in both translated kernels (input words 244-256,
reconstruction 592-604) and the previous frame's exposure as the input kernel's texture 352
(cuda_copy_exposure_kernel's half surface). The runtime left all of it at the model dump's values, so HDR games
with a pre-exposure other than 1 or a changing exposure measured 18-39 dB against NGX; now 54-64 dB.

AutoExposure (flag 0x40), which the shim refused for E/F: NGX runs K's log-mean measurement on M's grid every
second frame. exposure_k.comp MODE 3 writes it to the 1x1 image the kernels sample; sr/exposure_copy_e.comp
keeps the previous exposure. 60-66 dB over eleven frames at 960x540 -> 1920x1080, pre-exposure 19-21.

The reconstruction's jitter words 544-556 are (y, x); the runtime wrote (-x, -y), which matches only the
harness's first three jitters, so every later frame of a game's jitter sequence measured 26-28 dB. Twelve-frame
captures now measure 60-66 dB HDR and LDR; on the Steam Deck through the shim 51-67 dB against ZLUDA.

The D3D12 harness takes per-frame lists in D4R_HARNESS_PRE_EXPOSURE / _EXPOSURE_SCALE; d4r-e takes the
AutoExposure flag, per-frame exposure values and the game's exposure texture. E/F models need rebuilding
(the install's model cache key changes); older models keep the old binding and refuse AutoExposure.
This commit is contained in:
woutermans committed 2026-10-19 02:42:48 +02:00
1 parent 916b0d1467
commit 3f66081add
12 files changed
+249 -47

No files matched your search

+19 -7
View File
@@ -2,9 +2,10 @@
// after frame, and writes the output (and with D4R_E_DUMP_TENSORS=1 the network's tensors) after each frame.
//
// usage: d4r-e MODEL_DIR INPUT_DIR OUT_DIR
// INPUT_DIR: frames.txt ("size RW RH OW OH HDR LOWRES MW MH", then per frame "frame F jx jy mvsx mvsy reset") and
// INPUT_DIR: frames.txt ("size RW RH OW OH HDR LOWRES MW MH [AUTOEXPOSURE]", then per frame
// "frame F jx jy mvsx mvsy reset [preExposure exposureScale]") and
// F-color.bin (RGBA16F, render size), F-motion.bin (RG16F, MW x MH),
// F-depth.bin (R32F, render size, with LOWRES). D4R_E_TIME_REPEAT=N: best of N GPU timings per frame.
// F-depth.bin (R32F, render size, with LOWRES), optionally F-exposure.bin (R32F 1x1: the game's exposure, HDR). D4R_E_TIME_REPEAT=N: best of N GPU timings per frame.
#include "runtime.h"
#include <algorithm>
#include <cstdlib>
@@ -125,11 +126,16 @@ int main(int argc, char** argv) try {
auto framesText = read(inputDir + "/frames.txt");
std::istringstream frames(std::string(framesText.begin(), framesText.end()));
std::string key;
uint32_t rw, rh, ow, oh, hdr, lowRes, mw, mh;
frames >> key >> rw >> rh >> ow >> oh >> hdr >> lowRes >> mw >> mh;
uint32_t rw, rh, ow, oh, hdr, lowRes, mw, mh, autoExposure = 0;
std::string sizeLine;
std::getline(frames, sizeLine);
std::istringstream size(sizeLine);
size >> key >> rw >> rh >> ow >> oh >> hdr >> lowRes >> mw >> mh;
size >> autoExposure;
d4r::EModel model(c.physical, c.device, modelDir);
d4r::EEngine engine(model, rw, rh, ow, oh, hdr != 0, lowRes != 0);
d4r::EEngine engine(model, rw, rh, ow, oh, hdr != 0, lowRes != 0, autoExposure != 0);
Input color(c, VK_FORMAT_R16G16B16A16_SFLOAT, rw, rh), motion(c, VK_FORMAT_R16G16_SFLOAT, mw, mh), depth(c, VK_FORMAT_R32_SFLOAT, rw, rh);
Input exposure(c, VK_FORMAT_R32_SFLOAT, 1, 1);
HostBuffer staging(c, std::max<VkDeviceSize>(VkDeviceSize(ow) * oh * 8, 64u << 20));
VkQueryPoolCreateInfo qp{VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO};
qp.queryType = VK_QUERY_TYPE_TIMESTAMP; qp.queryCount = 4;
@@ -148,16 +154,22 @@ int main(int argc, char** argv) try {
barrier(cb, in.image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_GENERAL);
});
};
while (frames >> key) {
for (std::string line; std::getline(frames, line);) {
std::istringstream fields(line);
if (!(fields >> key)) continue;
uint32_t f; int reset;
d4r::EFrameSettings s;
frames >> f >> s.jitter[0] >> s.jitter[1] >> s.motionScale[0] >> s.motionScale[1] >> reset;
fields >> f >> s.jitter[0] >> s.jitter[1] >> s.motionScale[0] >> s.motionScale[1] >> reset;
if (!(fields >> s.preExposure >> s.exposureScale)) s.preExposure = s.exposureScale = 1;
s.reset = reset != 0;
upload(color, "color", f, rw, rh, 8);
upload(motion, "motion", f, mw, mh, 4);
if (lowRes) upload(depth, "depth", f, rw, rh, 4);
const bool gameExposure = std::ifstream(inputDir + "/" + std::to_string(f) + "-exposure.bin").good();
if (gameExposure) upload(exposure, "exposure", f, 1, 1, 4);
d4r::FrameImages images{};
images.color = color.view; images.motion = motion.view; images.depth = depth.view;
if (gameExposure) images.exposure = exposure.view;
engine.setFrame(images, s);
if (first) c.submit([&](VkCommandBuffer cb) { model.recordUpload(cb); engine.recordInitialize(cb); first = false; });
double best = 1e30, in = 0, net = 0, fin = 0;