kernels: compare two per-launch dumps byte for byte

The bridge's other instrumentation - D4R_CUDA_LAUNCH_STATS with D4R_CUDA_LAUNCH_DUMP_DIR - already dumps every allocation, surface object and texture object named in a launch's packed argument buffer, after the launch has finished. That is enough to check one reimplemented kernel on its own: the translated kernel's surface files are its output, and a native replacement has to reproduce them. What was missing was the comparison. compare_launch_dump.py keys the files by kernel, argument offset, kind, shape and format rather than by the global launch sequence, so a replacement that adds launches of its own does not break the alignment, and it reports the first differing byte per argument. Verified end to end today: cuda_dldn_engine_swin_enc0_kernel's PTX module was compiled offline with d4r_emit, served from a native directory (the bridge logs `[d4r-native] "cuda_dldn_engine_swin_enc0_kernel" -> ...hsaco`), and its run is byte-identical to the translated one across all 54 of its dumped arguments - weight image, activation buffer, four output surfaces. That also shows the offline build reproduces the loader's own compile exactly, which every native denoiser layer will rest on. docs/native-kernels.md gains the dump-and-diff step, which is the only way to localise a mismatch to one of the eleven layers that sample textures and write surfaces.
2026-10-12 12:47:53 +02:00
1 parent dbef6f794e
commit 708b158332
@@ -178,8 +178,10 @@ The release includes both fast and accuracy network sets. Full releases also inc
   ```
   It runs the prep and main kernels, saves every buffer after the first launch, and times 30 launches. The replay tool refuses captures with texture or surface objects; use the full harness for those kernels.
-3. **Compare against a reference.** `kernels/tools/pwin_model.py` (K) and `swin_model.py` (M) are numpy models of the layers, written stage by stage against `ptxsim.py`, a small vectorised PTX interpreter that runs NVIDIA's kernels on the CPU. A bit-exact variant should match the model within f16 rounding; after any change, compare the new build's buffers with the previous build's.
+3. **A per-layer oracle for kernels with texture and surface objects.** The replay capture above cannot describe those kernels (the replayers refuse them), but the bridge's other instrumentation can: with `D4R_CUDA_LAUNCH_STATS=1` and `D4R_CUDA_LAUNCH_DUMP_DIR=<dir>` every `cuLaunchKernel` is followed by a synchronize and a raw dump of every allocation, surface object and texture object named in its packed argument buffer, one file per launch and argument (`launch-NNN-<kernel>-argNNN-{buffer,surface,texture}-<shape>-fmtN.bin`). Run the harness once with the translated kernel and once with the native one and compare that kernel's files byte for byte: the surface files are the layer's own output, and a mismatch localises to that layer rather than to the frame. `cuda_dldn_engine_swin_enc0_kernel` dumps its weight image at arg040, its activation buffer at arg048 and four surfaces (1280x736x4 and 640x368x4 RGBA16F) as its outputs. The dump is large - 3 frames cost 3.7 GB - so keep the frame count small.
-4. **Check the whole pipeline.** Replay captured frames through the harness with both kernel sets and compare the outputs with `kernels/tools/psnr.py`. `D4R_CUDA_KERNEL_PROFILE=1` makes the bridge time every kernel; `kernels/tools/kprof.py` summarises the log per frame.
+
 4. **Compare against a reference.** `kernels/tools/pwin_model.py` (K) and `swin_model.py` (M) are numpy models of the layers, written stage by stage against `ptxsim.py`, a small vectorised PTX interpreter that runs NVIDIA's kernels on the CPU. A bit-exact variant should match the model within f16 rounding; after any change, compare the new build's buffers with the previous build's.
 5. **Check the whole pipeline.** Replay captured frames through the harness with both kernel sets and compare the outputs with `kernels/tools/psnr.py`. `D4R_CUDA_KERNEL_PROFILE=1` makes the bridge time every kernel; `kernels/tools/kprof.py` summarises the log per frame.
 ## Per-kernel cost
@@ -0,0 +1,100 @@
 #!/usr/bin/env python3
 """Compare two D4R_CUDA_LAUNCH_DUMP_DIR trees byte for byte.
 usage: compare_launch_dump.py A_DIR B_DIR [--kernel SUBSTR] [--quiet]
 The bridge writes one file per launch and per argument named
 `launch-NNN-<kernel>-argNNN-<kind>-<shape>-fmtN.bin` (wine_nvcuda_bridge.c
 summarize_resource). The launch number is a global sequence, so it is not a
 stable key: a native kernel that runs its own prep shifts every later number.
 Files are therefore keyed by (kernel, argument offset, kind, shape, format) and
 by how many times that key has occurred so far, which survives a shift as long
 as each kernel is launched the same number of times.
 Every dump is written after the launch returns, so a surface file is that
 layer's own output and a read-only argument (a weight image, a texture) is its
 input. Any difference in an output is the kernel's; identical read-only
 arguments are the check that the two runs are aligned at all.
 """
 import argparse
 import os
 import re
 import sys
 NAME = re.compile(r'^launch-(\d+)-(.+)-arg(\d+)-(buffer|surface|texture)-(.+)-fmt(-?\d+)\.bin$')
 # A buffer's shape string embeds the device pointer it was found at, which differs
 # between runs. Keep the offset within the allocation, drop the base address.
 BASE = re.compile(r'base0x[0-9a-f]+')
 def index(directory):
    """(kernel, arg, kind, shape, fmt, occurrence) -> (sequence, path)."""
    entries = {}
    counts = {}
    for name in sorted(os.listdir(directory)):
        match = NAME.match(name)
        if match is None:
            continue
        sequence, kernel, argument, kind, shape, fmt = match.groups()
        key = (kernel, int(argument), kind, BASE.sub('base', shape), int(fmt))
        occurrence = counts.get(key, 0)
        counts[key] = occurrence + 1
        entries[key + (occurrence,)] = (int(sequence), os.path.join(directory, name))
    return entries
 def first_difference(a, b):
    """(offset, count) of the differing bytes, or None when equal."""
    if len(a) != len(b):
        return None
    count = 0
    first = -1
    for index_ in range(len(a)):
        if a[index_] != b[index_]:
            if first < 0:
                first = index_
            count += 1
    return (first, count) if count else None
 def main(argv):
    parser = argparse.ArgumentParser(description=__doc__,
                                     formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument('a')
    parser.add_argument('b')
    parser.add_argument('--kernel', default='', help='only keys whose kernel name contains this')
    parser.add_argument('--quiet', action='store_true', help='report only keys that differ')
    args = parser.parse_args(argv[1:])
    left, right = index(args.a), index(args.b)
    keys = sorted(set(left) | set(right))
    mismatched = missing = compared = 0
    for key in keys:
        if args.kernel and args.kernel not in key[0]:
            continue
        label = '%s arg%03d %s %s fmt%d #%d' % key
        if key not in left or key not in right:
            print('MISSING %s: %s' % (label, 'only in A' if key in left else 'only in B'))
            missing += 1
            continue
        with open(left[key][1], 'rb') as handle:
            a = handle.read()
        with open(right[key][1], 'rb') as handle:
            b = handle.read()
        compared += 1
        difference = first_difference(a, b)
        if difference is None:
            if not args.quiet:
                print('ok      %s (%d bytes)' % (label, len(a)))
        else:
            offset, count = difference
            print('DIFFERS %s: %d of %d bytes, first at %d (A %02x B %02x)' %
                  (label, count, len(a), offset, a[offset], b[offset]))
            mismatched += 1
    print('%d arguments compared, %d differ, %d missing' % (compared, mismatched, missing))
    return 1 if mismatched or missing else 0
 if __name__ == '__main__':
    sys.exit(main(sys.argv))