Files
d4r/kernels/rr/rrswin_dec2_epilogues.md
woutermans 0fe3a28cb7 kernels/rr: staged-image Swin layer engine prototypes
Per-layer staged-image engine for the eleven translated denoiser Swin
layers: descriptors (verified offsets per phase), cut recipes that splice an
isolated native phase into NVIDIA's PTX, wave templates, epilogue notes and
GPU probes. The isolated phase-8 splice produces a byte-identical final
4K frame but is slower than the translated body (14.90 vs 6.59 ms) - a
correctness experiment, not a shipping improvement; docs/rr-engine.md
records the design and the next direct-converter win (B operands built
from transposed accumulators).
2026-10-12 12:47:53 +02:00

46 KiB
Raw Permalink Blame History

cuda_dldn_engine_swin_dec2_kernel — the epilogues of phases 1…38

Companion to kernels/rr/rr_layer_spec.py dec2. Everything below was read out of ~/.cache/d4r-rr-corpus/0021-PREPASS_ENTRYPOINT_NAME.ptx, entry cuda_dldn_engine_swin_dec2_kernel (15796 statements).

0. Conventions

  • sNNNN is a statement index in the linearisation rr_layer_spec.py documents (statements of the entry body; a wrapped { … } block and a 4-line mma.sync count as one statement, numbered at their first line). LNNNNN is the physical line of the same instruction in the corpus file, so every claim can be checked with sed -n 'LNNNNN,LNNNNN p' …ptx.
  • An epilogue is the run of statements between one phase's last mma and the next phase's first mma. Because phase 1 is the first GEMM, the prologue (s1–s451) doubles as phase 1's prelude and is described here too.
  • W denotes the prepared weight image: ld.param.u64 %rd1, [cuda_dldn_engine_swin_dec2_kernel_param_0+40] (s7, L1035) and cvta.to.global.u64 %rd32, %rd1 (s8, L1036). Every ld.weak.global.ca.v4.u32 reads W + 16*laneid + imm.
  • The C/D register layout and the weight-addressing formula are fixed (see the settled notes): C/D reg0 = (row g, cols 2t,2t+1), reg1 = (row g+8, same cols), g = laneid>>2, t = laneid&3.
  • Anything I could not pin down from the PTX is in §7 under uncharacterised.

1. Shared vocabulary

1.1 Addresses and parameters an epilogue touches

symbol how it is obtained statements
param_0 mov.b64 %rd7, …_param_0 s1, L1018
W (weight image) ld.param.u64 %rd1, [param_0+40] → cvta… %rd32, %rd1 s7/s8, L1035–L1036
extents (ex, ey) ld.param.v2.u32 {%r2147, %r2148}, [param_0+0] s13, L1041
%r7 = 8·ex, %r8 = ey·8·ex shl.b32 %r7, %r2147, 3 / mul.lo.s32 %r8, %r2148, %r7 s14/s15, L1042–L1043
block origins %r1, %r2 sub.s32 %r1, %r2142, %r2143 / sub.s32 %r2, %r2146, %r2144 s6/s12, L1034/L1040
%rd33 = param_0+8 ld.param.u64 %rd33, [param_0+8] s28, L1056
texture / skip input ld.param.u64 %rd2, [%rd7+24] = param_0+24 s2288, L4638
output plane arena ld.param.u64 %rd571, [%rd7+48] = param_0+48 → cvta… %rd4, %rd571 s15333/s15334, L36645–L36646
block-origin table ld.param.v2.u16 {%rs264, %rs265}, [param_0+80] s4, L1032
window index tid.z mov.u32 %r2350, %tid.z s378, L1408
shared arena .shared .align 4 .b8 _ZZ33…dec2_kernel…E4smem[9600] L1027 (declaration; not a statement)
per-lane weight address mul.wide.u32 %rd, %laneid, 16 + add.s64 %rd, W, %rd every load block, e.g. s11444–s11445, L24247–L24248
per-lane table address shl.b32 %r, %laneid, 2 → cvt.u64.u32 → and.b64 %rd, %r, 12 → add.s64 %rd, %rd32, %rd = W + ((laneid·4)&12) e.g. s12456–s12460, L27063–L27068

The last row matters: the ld.global.v2.u16 / ld.global.u32 table loads in an epilogue read W + ((laneid<<2)&12) + off, i.e. the same 16-byte slot is read by the four lanes with equal laneid&3, and consecutive loads pick consecutive 4-byte words inside one 64-byte record — so those tables are indexed by column, not by row.

Two per-tid.z strides exist, both in phase 1's prelude and nowhere else:

group base statements
phase 1's B W + 12288·tid.z + 16·laneid s379–s384, L1409–L1416
phase 1's C bias W + 192·tid.z + ((laneid·4)&12) + 49152…49328 s432–s439, L1512–L1521

%r2350·12288 (s379) is the only ×12288 in the file and %r2350·96 → ×2 (s432/s433) the only ×192, so no other phase's loads carry a window term.

1.2 Constants that survive between epilogues

register first definition f32/f64 literal value used (after the cvt to f16)
%f476 s7488 mov.f32 %f476, 0f3C8CCB50 0.017186790704727173 f16 0x2466 = 0.017181396484375
%fd383 s7492 mov.f64 %fd383, 0dBFE1CC0000000000 −0.55615234375 f16 0xB873
%fd385 s7497 mov.f64 %fd385, 0d3FE1CC0000000000 +0.55615234375 f16 0x3873
%f478 s7503 mov.f32 %f478, 0f3F6D6000 0.92724609375 f16 0x3B6B
%f480 s7508 mov.f32 %f480, 0f3FB00000 1.375 f16 0x3D80
%fd1 s6132 mov.f64 %fd1, 0d3F20000000000000 2⁻¹³ f16 0x0800 = 0.0001220703125
%f861 s12301 mov.f32 %f861, 0f3ED306EB 0.4121621549129486 f16 0x3698 = 0.412109375
%f862 s12304 mov.f32 %f862, 0f3DA60DD6 0.0810810774564743 f16 0x2D30 = 0.0810546875
%f863 s12307 mov.f32 %f863, 0f3F000000 0.5 f16 0x3800
%f864 s12310 mov.f32 %f864, 0f40000000 2.0 f16 0x4000

%f861…%f864 are materialised inside E14 (s12301–s12312) and only re-cvt by E16, E18, E20, …, E36 (s12642 cvt.rn.f16.f32 low, %f861, …). %fd1 is defined once, in E1 (s6132), and only re-cvt by E13.

Also live across the region, as in the sibling kernel: %r5908 = 1 (s5595, L8140) and %r5919 = -1 (s6074, L9586) supply the butterfly delta and the membermask of every shfl.sync.bfly.b32 in the epilogues (s6075 shfl.sync.bfly.b32 %r5842,%r5838,%r5908,%r5845,%r5919); the group/width operand is per-instruction.

1.3 Two quantisation helpers

  • cvt.rn.satfinite.e4m3x2.f16x2 — the requantisation used everywhere (f16x2_to_e4m3x2_satfinite_bits in ptx/lib/zluda_ptx_impl.cpp).
  • cvt.rn.f16x2.e4m3x2 — its inverse, the unpack, used only when a shared tile that was stored e4m3-packed is read back (E1: 96 of them, s5594–s5690, L8139–L8437).
  • rcp.approx.ftz.f32, rsqrt.approx.ftz.f32, ex2.approx.ftz.f32, div.approx.ftz.f32 — always wrapped as cvt.f32.f16 → op → cvt.rn.f16.f32 (round each step through f16).

2. Summary

epilogue range stmts named operation
E1 s596–s7088 6493 patch expand into shared + per-column RMS norm + gain + restage
E2 s7113–s7119 7 next-B weight loads (P3)
E3 s7144–s7150 7 next-B weight loads (P4)
E4 s7175–s7181 7 next-B weight loads (P5)
E5 s7206–s7212 7 next-B weight loads (P6)
E6 s7237–s7243 7 next-B weight loads (P7)
E7 s7268–s7438 171 score-bias C-seed loads (24×512 B) + 80 packs → P8's A and B
E8 s7487–s11329 3843 cubic-exponent softmax, V transpose, pack
E9 s11378–s11546 169 pack P9's D → shared, per-column table → P10's C, next B
E10 s11559–s11577 19 1 shared A fragment + 6 next-B loads (P11)
E11 s11590–s11608 19 1 shared A fragment + 6 next-B loads (P12)
E12 s11621–s11639 19 1 shared A fragment + 6 next-B loads (P13)
E13 s11652–s12287 636 RMS norm of P13's D + gain, packs → P14's A and P16's A, next B/C
E14 s12300–s12556 257 clamped-cubic activation of P14's D, per-column table + residual → P15's C
E15 s12569–s12628 60 24 packs → P16's A, next B, P16's C bias
E16 s12641–s12804 164 clamped-cubic activation of P16's D → P17's A, next B
E17 s12817–s12876 60 24 packs → P18's A, next B, P18's C bias
E18 s12889–s13052 164 activation of P18's D → P19's A, next B
E19 s13065–s13124 60 24 packs → P20's A, next B, P20's C bias
E20 s13137–s13300 164 activation of P20's D → P21's A, next B
E21 s13313–s13372 60 24 packs → P22's A, next B, P22's C bias
E22 s13385–s13548 164 activation of P22's D → P23's A, next B
E23 s13561–s13620 60 24 packs → P24's A, next B, P24's C bias
E24 s13633–s13796 164 activation of P24's D → P25's A, next B
E25 s13809–s13868 60 24 packs → P26's A, next B, P26's C bias
E26 s13881–s14044 164 activation of P26's D → P27's A, next B
E27 s14057–s14116 60 24 packs → P28's A, next B, P28's C bias
E28 s14129–s14292 164 activation of P28's D → P29's A, next B
E29 s14305–s14364 60 24 packs → P30's A, next B, P30's C bias
E30 s14377–s14540 164 activation of P30's D → P31's A, next B
E31 s14553–s14612 60 24 packs → P32's A, next B, P32's C bias
E32 s14625–s14788 164 activation of P32's D → P33's A, next B
E33 s14801–s14860 60 24 packs → P34's A, next B, P34's C bias
E34 s14873–s15036 164 activation of P34's D → P35's A, next B
E35 s15049–s15108 60 24 packs → P36's A, next B, P36's C bias
E36 s15121–s15284 164 activation of P36's D → P37's A, next B
E37 s15297–s15504 208 pack → 12 guarded plane-arena scatter → B/C for P38
E38 s15511–s15796 286 4 D stores → shared, row max, softmax, 9+2 texture reads, surface store

3. The epilogues

E1 — s596–s7088: patch expand, RMS norm, gain, restage

6493 statements; op histogram (top): 1103 add.s32, 477 shl.b32, 446 mov.b32, 413 $label, 409 sub.s32, 355 min.u32, 288 mul.f16, 120 cvt.rn.satfinite.e4m3x2.f16x2, 96 cvt.rn.f16x2.e4m3x2, 96 rsqrt.approx.ftz.f32, 42 st.shared.* (34 u32 + 2 v2.u16 + 6 v4.u32), 42 ld.shared.* (24 v2.u16 + 18 v4.u32), 36 ld.global.v2.u16, 8 shfl.sync.bfly.b32, 1 ld.param, 2 ld.weak….

Sub-sections (bounded by the first/last statement of each op):

range what
s596–s702 the head packs of phase 1's D (72 cvt.rn.satfinite…), → %r9…%r41
s716–s1478 per-lane index arithmetic + the 36 guarded patch stores (34 st.shared.u32 s770–s1460 + 2 st.shared.v2.u16 s743/s1478); 1st bar.sync at s1480
s1524–s2287 24 ld.shared.v2.u16 read back of the packed patch
s2288 ld.param.u64 %rd2, [%rd7+24]: the patch-gather buffer
s2444–s5593 24 clamped ld.global.v2.u16 gathered from that buffer
s5594–s5690 96 cvt.rn.f16x2.e4m3x2 unpacks of the read-back patch
s5693–s6276 the sum trees (196 add.f16 exact + 52 add.f16x2)
s6132 the 2⁻¹³ constant %fd1 (s6132 mov.f64 %fd1, 0d3F20000000000000)
s6075–s6128 the 8 shfl.sync.bfly.b32 (first at s6075)
s6282–s6659 96 rsqrt.approx.ftz.f32, each cvt.f32.f16 → rsqrt → cvt.rn.f16.f32
s6668–s6679 gain loads W + ((laneid·4)&12) + {49920…50096} (12 ld.global.v2.u16)
s5693–s6966 the 288 mul.f16 (two groups: inv · gain then x · (inv·gain))
s7008–s7054 24 packs → the 6 st.shared.v4.u32 at s7008/7021/7034/7044/7049/7054
s7059–s7082 bar.sync (s7059) + 18 ld.shared.v4.u32 (s7064–s7082) → phases 2…7's A
s7083–s7088 2 ld.weak (s7086/s7088) → phase 2's B

Evidence for the patch scatter (36 stores):

s743  L2836: st.shared.v2.u16 [%r2373], {%rs192, %rs193}
s770  L2865: st.shared.u32 [%r2386], %r9
s800  L2897: st.shared.u32 [%r2403], %r10
s1478 L3641: st.shared.v2.u16 [%r2787], {%rs262, %rs263}
s7008 L11717: st.shared.v4.u32 [%r6091], {%r6095, %r6094, %r6093, %r6092}
s7044 L11788: st.shared.v4.u32 [%r6110+6144], {%r6114, %r6113, %r6112, %r6111}
s7054 L11798: st.shared.v4.u32 [%r6110+7168], {%r6122, %r6121, %r6120, %r6119}

Each patch store is guarded exactly as in the sibling kernel (setp.gt.u32 %p…, %r…, 9 / or.pred / @%p bra …; the same (row·40 + col·8 + t)·4 index form — cf. rrswin_dec1_epilogues.md §3/E1, which carries the same 400-block emitter). The second staging area is at [%r6091], [%r6091+512], [%r6091+1024], [%r6110+6144], [%r6110+6656], [%r6110+7168], where %r6091 = smem + tid.z·2048 (s6994 shl.b32 … %r2350, 11) and %r6110 = smem + tid.z·3072 (s7040).

The norm:

s6132 L9664:  mov.f64 %fd1, 0d3F20000000000000        // 2^-13
s5693 L8430:  add.f16 %rs456,%rs457,%rs458
s6282 L10110: rsqrt.approx.ftz.f32 fl, fl
s6668 L10735: ld.global.v2.u16 {%rs1959, %rs1960}, [%rd247+49936]
s6681 L10749: mul.f16 %rs…, %rs…, %rs…                // inv * gain

There is no mean subtraction and no 1/N. The accumulated quantity is the raw sum of squares of the 3 patch channels plus the constant 2⁻¹³, and the norm is applied per column (the gain is read at W + ((laneid<<2)&12) + off, so it is constant across rows).

Inputs: phase 1's D (%r593…%r826, 12 fragments — phase 1 has nA = 12), W, param_0+8, param_0+24, %r1/%r2/%r7/%r8/%r2147/%r2148, %r2350 (tid.z). Outputs (live past s7088): the patch-expand st.shared regions, the 18 ld.shared.v4.u32 A fragments %r18995…%r19066, phase 2's B %r6125…%r6132, the 4 constants of §1.2 (%fd1, %f861…%f864 are not yet defined here).

E2…E6 — s7113–s7243: next-B loads

Five 7-statement blocks of identical shape:

s7113 L12005: mov.u32 %r6373, %laneid
s7114 L12007: mul.wide.u32 %rd…, %r6373, 16
s7115 L12008: add.s64 %rd…, %rd2?, %rd…
s7117 L12017: ld.weak.global.ca.v4.u32 { %r6374,…%r6377},[%rd250]
s7119 L12021: ld.weak.global.ca.v4.u32 { %r6378,…%r6381},[%rd251]
epilogue weight block destination phase
E2 50112…50636 P3's B
E3 53184…53708 P4's B
E4 51136…51660 P5's B
E5 54208…54732 P6's B
E6 52160…52684 P7's B

The weight sub-image is laid out P2, P4, P6, P3, P5, P7 (blocks of 1024 bytes at W + 50112 + 1024·k), not in phase order.

E7 — s7268–s7438: score-bias C-seed loads + packs → P8's A and B

171 statements, no ld.global outside the two groups below.

  • 24 loads W + 16·laneid + 56256 + 512·i, i = 0…23, at s7272…s7318 (L12922…L13014) into %r7619…%r7714; the last block's add.s64 is s7317 add.s64 %rd283, %rd467, 17920. Byte block [56256, 68544) = 24 × 512 = 12288 bytes — phase 8's C score-bias seed (the phase table prints the last slot offset, 68044).
  • 80 packs / 40 b32, s7319–s7438, whose sources are registers written by phases 3…7's mma (%r7129/%r7149/…, %r7229/…, %r7360…). The two destinations are %r8161…%r8272 (phase 8's B) and %r7907…%r8030 (phase 8's A), e.g.
s7320 L13020: cvt.rn.satfinite.e4m3x2.f16x2 %rs1984, %r7149
s7321 L13022: mov.b32 %r8161, {%rs1983, %rs1984}
s7391 L13185: cvt.rn.satfinite.e4m3x2.f16x2 %rs2032, %r7149
s7393 L13190: mov.b32 %r7907, {%rs2031, %rs2032}

s7439 then consumes them: mma … {A=%r7907,%r7908,%r7909,%r7910}, {B=%r8161,%r8162}, {C=%r7619,%r7620}. %r7149 is phase 3's first D (s7089–s7112) and %r7129/%r7360 likewise belong to phases 3/4/6/7, so both of phase 8's operands are packs of the six preceding projections.

E8 — s7487–s11329: cubic-exponent softmax, V transpose, pack

Byte-identical in kind to cuda_dldn_engine_swin_enc0_kernel's E5 (see rrswin_enc0_epilogues.md §3/E5): 1048 mov.b32, 672 cvt.rn.f16.f32, 384 cvt.rn.f16.f64, 192 each of fma.rn.f16x2 / cvt.f32.f16 / rcp.approx.ftz.f32 / mul.f16, 144 packs, 48 movmatrix, 16 shfl.sync.bfly.b32, 96 each of max/min/neg/abs.f16x2.

Sub-sections:

range what
s7488–s7513 the 5 constants (§1.2 of the sibling file): s7488 0f3C8CCB50, s7492/s7497 the f64 clamps, s7503/s7508 the cubic pair
s7514–s9604 96 per-element groups (scale, clamp, cubic, and.b32 …,2145419232)
s9605–s9690 in-lane partial sums (add.f16x2)
s9691–s9820 16 shfl.sync.bfly → 8 row sums
s9821–s10582 192 rcp.approx.ftz.f32 → 96 reciprocals
s10583–s11065 192 mul.f16 → 96 probabilities
s11066–s11113 48 movmatrix.sync.trans.aligned.m8n8.b16 of phase 7's D
s11114–s11328 144 packs

Evidence:

s7514 L13706: and.b32 %r…, %r…, 2145419232               // 0x7FE07FE0
s9691 L20503: shfl.sync.bfly.b32 %r10441,%r10437,%r5908,%r10444,%r5919
s9821 L20829: rcp.approx.ftz.f32 fl, fl
s11066 L23128: movmatrix.sync.trans.aligned.m8n8.b16 %r10801, %r7378
s11114 L23272: cvt.rn.satfinite.e4m3x2.f16x2 %rs3432, %r10809

%r10801 is the first movmatrix result and its source %r7378 is phase 7's first D (s7244–s7267); the movmatrix sources run over phase 7's 48 D registers. The transposed results become phase 9's B (%r11407 ← mov.b32 {%rs3431,%rs3432} at s11116 after the pack at s11114), and the probability packs (%r11113 ← mov.b32 {%rs3479,%rs3480} at s11188) become phase 9's A.

The per-element function is the same cubic-exponent approximation as enc0's E5 (no row maximum, no bias add):

m  = clamp(f16(x * 0.017181396484375), -0.55615234375, +0.55615234375)
u  = f16(1.375 + m*(0.92724609375 - m*m))
expval = f16 from bits  ((u_bits << 5) & 0x7FE07FE0)

Memory: none (0 ld.*, 0 st.* in 3843 statements).

E9 — s11378–s11546: pack P9's D → shared, per-column table → P10's C

169 statements: 32 packs, 8 add.f16, 6 ld.weak, 12 ld.global.v2.u16, 4 st.shared.v4.u32, 1 ld.shared.v4.u32.

s11378 L24111: bar.sync 0
s11384 L24120: cvt.rn.satfinite.e4m3x2.f16x2 %rs3576, %r11141
s11396 L24147: st.shared.v4.u32 [%r19198], {%r19202, %r19201, %r19200, %r19199}
s11435 L24234: st.shared.v4.u32 [%r19198+1536], {%r19214, %r19213, %r19212, %r19211}
s11436 L24235: bar.sync 0
s11442 L24243: ld.shared.v4.u32 {%r19219,…%r19222}, [%r19218]
s11446 L24249: add.s64 %rd284, %rd469, 68544
s11451 L24259: add.s64 %rd286, %rd469, 69568
s11463 L24280: ld.global.v2.u16 {%rs4886, %rs4887}, [%rd473+136144]
s11475 L24293: add.f16 %rs3610,%rs4909,%rs459
s11477 L24300: mov.b32 %r11589, {%rs3607, %rs3610}
  • 32 packs of phase 9's D → 16 b32 → four st.shared.v4.u32 at [%r19198 + {0,512,1024,1536}]; the bar.sync pair makes them visible, then one ld.shared.v4.u32 gives the first A fragment of phase 10.
  • 6 ld.weak at W + 16·laneid + {68544, 69056, 69568, 70080, 70592, 71104} → phase 10's B.
  • 12 ld.global.v2.u16 at W + ((laneid·4)&12) + {136128…136304} — a per-column vector (12 words per lane = 96 columns).
  • 48 add.f16 adding that vector to register pairs, producing the 24 registers %r11589 %r11590 %r11599 %r11600 … = phase 10's C (confirmed at s11547: C = {%r11589, %r11590}).

Which registers the table is added to is ambiguous in this file. The operands are %rs456/%rs459 and their 22 siblings, and the only textual definition of %rs456/%rs459 is at s5693/s5694 (L8430/L8434) inside E1 (the E1 sum-of-squares tree); nothing between s5694 and s11475 writes them. So either E9's residual is genuinely carried from E1, or the printed PTX reuses a register name across two disjoint live ranges without a rename. See §7/U1.

E10/E11/E12 — s11559–s11639: one shared A fragment + next-B loads

Three 19-statement blocks:

s11562 L24597: ld.shared.v4.u32 {%r19226,…%r19229}, [%r19225+2048]
s11566 L24603: add.s64 %rd290, %rd475, 90048
s11571 L24613: add.s64 %rd292, %rd475, 91072
epilogue shared A offset weight block destination phase
E10 +2048 90048…92620 P11's A, B
E11 +4096 111552…114124 P12's A, B
E12 +6144 133056…135628 P13's A, B

Each takes its single A fragment from the same shared staging region E9 wrote (offsets 2048/4096/6144 relative to %r19216) and reads 6 consecutive 512-byte weight fragments. All three phases take C = 0.

E13 — s11652–s12287: RMS norm of P13's D + gain

636 statements; 182 mov.b32, 144 mul.f16, 76 add.f16, 26 add.f16x2, 48 rsqrt.approx.ftz.f32, 4 shfl.sync.bfly, 24 packs, 12 ld.global.v2.u16.

range op
s11654–s11749 unpack P13's D halves (%r12019…) and square them (48 mul.f16, s11655–s11749)
s11750–s11804 in-lane add.f16x2 partial sums
s11776–s11794 4 shfl.sync.bfly.b32
s11875–s11878 add.f16 of the halves, then + 2⁻¹³ (%fd1, defined at s6132)
s11883–s12068 48 rsqrt.approx.ftz.f32
s12072–s12083 gain loads W + ((laneid·4)&12) + {136336, 136352, …, 136496, 136320} (12 ld.global.v2.u16)
s12085–s12226 96 mul.f16 — inv · gain then x · (inv·gain)
s12231–s12242 next-B loads W + {136512, 137024, 137536, 138048, 138560, 139072} → phase 14's B
s12248–s12251 ld.global.u32 at W + ((laneid·4)&12) + {139584, 139600, 139616, 139632} → phase 14's C bias
s12252–s12287 24 packs → %r12414…%r12457, phase 14's A

At s12157/s12158 the same epilogue also emits a second scaled copy of P13's D:

s12157 L26127: mul.f16 %rs4190,%rs4360,%rs4046
s12158 L26130: mov.b32 %r18435, {%rs4190, %rs4193}

%rs4360 is the unpack of P13's first D (s11654). The 24 registers %r18435…%r18458 are re-used by every even phase's A pack (E15 s12594, E17 s12842, E19 …, and E37 s15297's neighbours), i.e. phases 14, 16, 18, …, P36 all share one A operand derived from phase 13.

E14 — s12300–s12556: activation of P14's D, table + residual → P15's C

257 statements. Three things happen:

  1. s12301–s12440: the clamped-cubic activation on the 8 f16x2 registers %r12382 %r12392 %r12383 %r12393 %r12442 %r12452 %r12443 %r12453 (phase 14's D). This is the epilogue that materialises %f861…%f864 (s12301–s12312):
s12313 L26578: neg.f16x2 %r12467,%r12466
s12314 L26582: max.f16x2 %r12469,%r12382,%r12467
s12315 L26586: min.f16x2 %r12472,%r12469,%r12466
s12316 L26590: abs.f16x2 %r12475,%r12472
s12317 L26594: mul.f16x2 %r12477,%r12464,%r12475
s12318 L26598: sub.f16x2 %r12480,%r12463,%r12477
s12319 L26602: mul.f16x2 %r12483,%r12472,%r12480
s12320 L26606: add.f16x2 %r12486,%r12465,%r12483
s12321 L26610: mul.f16x2 %r12489,%r12382,%r12486

Per element: y = clamp(x,−2,+2), g = 0.5 + y·(0.412109375 − 0.0810546875·|y|), out = x·g — the same cubic ramp as the sibling kernel's E10. Reproduce the formula, not SiLU/GELU. 2. s12441–s12472: next-B and table loads. Six ld.weak at W + {139648, 140160, 140672, 141184, 141696, 142208} → phase 15's B; and 12 ld.global.v2.u16 at W + ((laneid·4)&12) + {142720…142896}. 3. s12473–s12556: the residual/table add and the packs. 48 add.f16 (add.f16 %rs4361,%rs4957,%rs4363 at s12473) combine the table with the 24 registers whose halves come from s11654 L24951: mov.b32 {%rs4360, %rs4363}, %r12019 — phase 13's D. The 24 results %r12737…%r12848 are phase 15's C; the 8 packs at s12545–s12556 give phase 15's A.

So phase 15 computes D15 = act(D14)·B15 + (D13 + table@142720).

E15 — s12569–s12628: packs of the shared A, next B, P16's C bias

60 statements:

s12572 L27414: add.s64 %rd320, %rd494, 142912
s12589 L27445: ld.global.u32 %r12908, [%rd497+145984]
s12593 L27450: cvt.rn.satfinite.e4m3x2.f16x2 %rs4511, %r18436
s12594 L27453: cvt.rn.satfinite.e4m3x2.f16x2 %rs4510, %r18435
  • six ld.weak at W + {142912, 143424, 143936, 144448, 144960, 145472} → phase 16's B;
  • four ld.global.u32 at W + ((laneid·4)&12) + {145984, 146000, 146016, 146032} → phase 16's C bias;
  • 24 packs of %r18435…%r18458 — the shared A that E13 produced — so phase 16's A is bit-identical to phase 14's… (checked: %r12971 ← mov.b32 {%rs4510, %rs4511} at s12595 and %r12971 is phase 16's first A register; contrast phase 14's first A register %r12414 ← mov.b32 {%rs4334, %rs4335} at s12254, sourced from %r18435 at s12253 — the same register, so yes, identical).

E16/E18/…/E36 — the 164-statement activation epilogues

Twelve shape-identical 164-statement epilogues (shape hash 706fcc703f68): 36 mov.b32, 32 cvt.rn.f16.f32, 24 mul.f16x2, 8 each neg/max/min/abs/sub/add.f16x2, 6 ld.weak, no ld.global, no constants of their own (they re-cvt %f861…%f864).

Each applies the E14 activation to its own phase's D and packs 8 b32 → the next odd phase's A, and reads 6 weight fragments → the next odd phase's B. Evidence (E16):

s12642 L27622: cvt.rn.f16.f32 low, %f861
s12651 L27649: max.f16x2 %r13026,%r12939,%r13024      // phase 16's D
s12658 L27677: mul.f16x2 %r13046,%r12939,%r13043
s12782 L28107: ld.weak.global.ca.v4.u32 { %r13253,…},[%rd326]   // W+146048
s12795 L28135: mov.b32 %r13397, {%rs4534, %rs4535}    // phase 17's A

No table add and no residual: these epilogues produce only A and B.

E17/E19/…/E37-pre — the 60-statement pack epilogues

Eleven shape-identical 60-statement epilogues (shape hash a7b578456a5a): 24 packs, 12 mov.b32, 6 ld.weak, 4 ld.global.u32.

s12594 L27453: cvt.rn.satfinite.e4m3x2.f16x2 %rs4510, %r18435
s12842 L28285: cvt.rn.satfinite.e4m3x2.f16x2 %rs4542, %r18435

Each packs the same 24 registers %r18435…%r18458 (so every even phase P16, P18, …, P36 gets the identical A operand), reads 6 weight fragments → the next even phase's B, and 4 ld.global.u32 → the next even phase's C bias (E15→P16@145984, E17→P18@152192, E19→P20@158400, E21→P22@164608, E23→P24@170816, E25→P26@177024, E27→P28@183232, E29→P30@189440, E31→P32@195648, E33→P34@201856, E35→P36@208064).

Shape of the whole MLP region (E13–E37)

E13  normed tile  → P14's A (s12252)      ; shared tile T = %r18435… (s12158)
P14: A = f(D13),  B = W14, C = bias@139584  → D14   (48×96×96)
E14: act(D14) → P15's A ; P15's C = D13 + table@142720
P15: A = act(D14), B = W15, C = D13+table   → D15   (16×96×32)
E15: pack T → P16's A ; B = W16 ; P16's C = bias@145984
P16: A = pack(T),  B = W16, C = bias@145984 → D16   (48×96×96)
E16: act(D16) → P17's A ; B = W17
P17: A = act(D16), B = W17, C = D15         → D17   (16×96×32)
E17: pack T → P18's A ; B = W18 ; P18's C = bias@152192
P18: A = pack(T),  B = W18, C = bias        → D18
E18: act(D18) → P19's A ; P19's C = D17
 …   (the same two-phase motif repeats with the bias offsets of §2 until)
P37: A = act(D36), B = W37, C = D35         → D37
E37: pack D37 → 12 plane-arena stores ; P38: B = W38, C = bias@212736
P38: A = those packs, B = W38, C = bias     → D38   (48×48×96)

The claim "odd phase's C = previous odd phase's D" is machine-checked for every odd phase from P17 on (P15 is the exception, whose C is the register pair E14 builds). The claim "even phase's C comes from a ld.global.u32" holds for P14…P38.

E37 — s15297–s15504: pack → 12 guarded plane-arena stores → B/C for P38

208 statements; the only st.global in the entry (12 of them, all here).

(a) requantise, s15297–s15332. 12 groups of 2 cvt + 1 mov.b32 → the 12 b32 %r481…%r492 (phase 37's D).

(b) plane-arena scatter, s15333–s15482.

s15333 L36645: ld.param.u64 %rd571, [%rd7+48]        // = param_0+48
s15336 L36650: shr.u32 %r493, %r18989, 2             // g = laneid>>2
s15337 L36651: and.b32 %r494, %r18989, 3             // t = laneid&3
s15340 L36654: add.s32 %r19271, %r480, %r19269       // y = %r480 + (laneid>>5)
s15346 L36660: add.s32 %r19277, %r1, %r19270         // x = %r1 + (g&7)
s15351 L36665: shl.b32 %r19278, %r19277, 3           // x<<3
s15352 L36666: mad.lo.s32 %r495, %r7, %r19271, %r19278
s15354 L36668: selp.b32 %r496, -1, %r19279, %p1
s15359 L36674: st.global.u32 [%rd573], %r481

Stride register %r7 = 8·ex (s14). Each store is idx = 8·ex·y + 8·x + d, d = t or t|4, guarded by (y<0)|(y≥ey)|(laneid>63)|(x<0)|(x≥ex) (s15341–s15350, L36655–L36664); a guard failure stores -1 as the index, which the following setp.lt.s32 + bra turns into "skip the store".

Store values, bases and column offsets (g = laneid>>2, t = laneid&3, %r480 = %r2 + 2·tid.z at s11653/L24950, %r8 = ey·8·ex):

store stmt value index base d
1 s15359 %r481 idx1 t
2 s15368 %r482 idx1 t|4
3 s15401 %r483 idx2 t
4 s15409 %r484 idx2 t|4
5 s15419 %r485 idx1 + %r8 t
6 s15427 %r486 idx1 + %r8 t|4
7 s15437 %r487 idx2 + %r8 t
8 s15445 %r488 idx2 + %r8 t|4
9 s15455 %r489 idx1 + 2·%r8 t
10 s15463 %r490 idx1 + 2·%r8 t|4
11 s15473 %r491 idx2 + 2·%r8 t
12 s15481 %r492 idx2 + 2·%r8 t|4

with the two bases built as mad.lo.s32 %rBASE, 8·ex, y, 8·x:

base stmt y x
idx1 = %r495 s15352, L36666 %r480 + (laneid>>5) %r1 + (g&7)
idx2 = %r499 s15394, L36712 %r19505 + ((g+8)>>3) %r1 + ((g+8)&7)

%r19505 is a second row origin: s15370 mov.u32 %r19511, %ctaid.y / s15372 shl.b32 %r19509, %r19511, 3 / s15375 sub.s32 %r19506, %r19509, %r19510 / s15376 add.s32 %r19505, %r19506, %r19507 with %r19507 = 2·tid.z (s15373/s15374), i.e. ctaid.y·8 − origin_y + 2·tid.z — so the two halves of this scatter start at different rows. The +%r8 (s15412 mad.lo.s32 %r502, %r2148, %r19493, %r495) and +2·%r8 (s15466 mad.lo.s32 %r511, %r2148, %r19496, %r505) forms then step one and two rows further.

(c) next B/C, s15483–s15504.

s15492 L36832: add.s64 %rd596, %rd600, 211200
s15493 L36834: ld.weak.global.ca.v4.u32 { %r19307,…%r19310},[%rd596]
s15497 L36842: ld.weak.global.ca.v4.u32 { %r19315,…%r19318},[%rd598]
s15503 L36851: ld.global.u32 %r19329, [%rd604+212736]
s15504 L36852: ld.global.u32 %r19339, [%rd604+212752]

Three weight fragments at W + {211200, 211712, 212224} → phase 38's B, two u32 at W + ((laneid&3)·4 &12) + {212736, 212752} → phase 38's C bias.

E38 — s15511–s15796: the final epilogue

286 statements; it consumes phase 38's four D accumulators (%r19360 %r19370 %r19497 %r19498) and writes the surface.

(a) staging, s15511–s15542.

s15511 L36895: bar.sync 0
s15522 L36908: st.shared.u32 [%r520], %r19360
s15526 L36913: st.shared.u32 [%r520+16], %r19370
s15539 L36927: st.shared.u32 [%r522], %r19361
s15541 L36930: st.shared.u32 [%r522+16], %r19371

Four guarded stores of the phase-38 D pairs into shared, indexed from %r518 = laneid>>2, %r519 = laneid&3 and %tid.z.

(b) coordinates and bounds check, s15543–s15570.

s15551 L36941: mov.u32 %r19403, %tid.x
s15559 L36949: add.s32 %r525, %r1, %r523             // x
s15560 L36950: add.s32 %r526, %r19512, %r524         // y
s15565 L36955: ld.param.v2.u32 {%r19418, %r19419}, [%rd634]
s15570 L36960: @%p412 bra $L__BB1_486

(c) row max over 5 shared values, s15571–s15591.

s15577 L36968: ld.shared.u32 %r19421, [%r19470]
s15583 L36974: ld.shared.u32 %r19442, [%r19470+16]
s15585 L36977: max.f16x2 %r19420,%r19421,%r19422
s15590 L36994: max.f16 %rs4961,%rs4958,%rs4963
s15592 L36999: sub.f16x2 %r19429,%r19421,%r19443

Five u32 (= 10 f16) per row, reduced in registers — no warp shuffle.

(d) softmax, s15592–s15676.

s15595 L37007: mul.ftz.f32 %f905, %f865, 0f3FB8AA3B   // * log2e
s15596 L37008: ex2.approx.ftz.f32 %f867, %f905
s15651 L37126: add.ftz.f32 %f922, %f920, %f882        // running row sum
s15653 L37128: div.approx.ftz.f32 %f883, %f904, %f922 // 1/rowsum, %f904 = 1.0

y = 2^((x−max)·log2e), p = y·(1/Σy) in f16, five f16x2 registers.

(e) 3×3 colour texture, s15677–s15758.

s15677 L37176: ld.param.v2.f32 {%f924, %f925}, [%rd635+136]
s15683 L37182: ld.param.u64 %rd622, [%rd635+88]
s15685 L37185: tex.base.2d.v4.f16.f32 {%rs4986,%rs4987,%rs4988,%rs4989}, [%rd606, {%f884,%f885}]
s15687 L37189: fma.rn.f16 %rs4990,%rs4986,%rs5000,%rs4983

%rd635 = param_0 (s15571), so the colour texture handle is param_0+88 and the coordinate scales are param_0+136. Nine tex.base.2d.v4.f16.f32 (s15685, 15697, 15709, 15721, 15726, 15732, 15744, 15750, 15755) cover the 3×3 neighbourhood of clamped (x−1, x, x+1) × (y−1, y, y+1), accumulated into 3 channels with fma.rn.f16.

(f) the two integer-coordinate textures, s15759–s15762.

s15759 L37357: ld.param.u64 %rd624, [%rd635+96]
s15760 L37359: tex.base.2d.v4.f16.s32 {%rs5130,%rs5131,%rs5132,%rs5133}, [%rd624, {%r525,%r526}]
s15761 L37361: ld.param.u64 %rd626, [%rd635+104]
s15762 L37363: tex.base.2d.v4.f16.s32 {%rs5134,%rs5135,%rs5136,%rs5137}, [%rd626, {%r525,%r526}]

These are the +96 / +104 texture reads the brief asks about — handles at param_0+96 and param_0+104, sampled at the integer pixel coordinate (%r525, %r526).

(g) combine, sigmoid gate, surface store, s15763–s15796.

s15763 L37366: cvt.f32.f16 %f902, %rs5138
s15764 L37369: mul.ftz.f32 %f935, %f902, 0fBFB8AA3B
s15765 L37370: ex2.approx.ftz.f32 %f936, %f935
s15766 L37371: add.ftz.f32 %f937, %f936, 0f3F800000
s15767 L37372: div.approx.ftz.f32 %f903, %f904, %f937    // 1/(1+2^(-log2e·v))
s15770 L37382: sub.f16 %rs5141,%rs5140,%rs5139           // 1 - sigmoid
s15771 L37386: mul.f16 %rs5144,%rs5118,%rs5139           // acc * sigmoid
s15772 L37390: fma.rn.f16 %rs5187,%rs5130,%rs5141,%rs5144// + tex96 * (1-sig)
s15777 L37410: set.nan.f16.f16 %rs5165,%rs5134,%rs5134   // + tex104 unless NaN
s15793 L37449: mov.u16 %rs5186, 0
s15794 L37451: sust.b.2d.v4.b16.zero [%rd629, {%r19489,%r526}], {%rs5187,%rs5188,%rs5189,%rs5186}

%rd629 = param_0+112 (s15582). The store column is %r525·8 (s15792 shl.b32 %r19489, %r525, 3), the row %r526; the fourth channel is the zero literal, so this is a 3-channel write into a 4-channel surface. The set.nan/bra triad (s15777–s15790) adds the +104 read only when it is not NaN.

Inputs: the 4 returned D registers, param_0+{48,88,96,104,112,136}, the shared scratch, constants 0f3FB8AA3B, 0fBFB8AA3B, 0f3F800000. Outputs: none (the kernel returns, s15796).


4. Texture reads, surface writes, and the +24 input

what where
param_0+8 — input feature plane arena, read by phase 1's A loads prologue s66…s451 (60 ld.global)
param_0+24 — the patch-gather source E1 reads s2288 (ld.param.u64 %rd2, [%rd7+24]) and the 24 ld.global.v2.u16 at s2444…s5593
param_0+48 — the plane arena E37 scatters phase 37's output to s15333, 12 st.global.u32 at s15359…s15481
param_0+88 — the colour texture handle s15683, 9 tex.base.2d.v4.f16.f32 at s15685…s15755
param_0+96 — texture handle #2 (integer coords) s15759/s15760
param_0+104 — texture handle #3 (integer coords) s15761/s15762
param_0+112 — the output surface s15582, sust.b.2d.v4.b16.zero s15794
param_0+136 — two f32 texture coordinate scales s15677

The skip input at +24 supplies the patch-gather source. E1 reads it once per lane per stencil position and never writes it; the buffer is addressed in 4-byte elements (one f16x2 = 2 channels) with index clamp(row)·8·ex + clamp(col)·8 + clamp(t,0,7):

s2318 L4671: mad.lo.s32 %r3078, %r3068, %r7, %r3077   // clamp_row*(8*ex) + clamp_col*8
s2440 L4800: add.s32 %r19540, %r2996, %r2992          // + clamp(t, - , 7)
s2442 L4803: mul.wide.u32 %rd191, %r19540, 4
s2444 L4805: ld.global.v2.u16 {%rs314, %rs315}, [%rd192]

The same index form and the same 24-gather shape as the sibling kernel's §3/E1.

The surface writes are param_0+112 (E38, s15794), a 4-channel b16 surface written with 3 live channels. The plane-arena writes are E37's 12 st.global.u32 to param_0+48 (s15359…s15481).

Nothing reads param_0+48 back in this entry: param_0+48 is the source of exactly one ld.param (s15333) and never the base of a load. Likewise the texture handles at +88/+96/+104 are only ever tex.* operands.

5. The phase table, M/N/K checked

rr_layer_spec.py dec2's phase table with M/16 · N/8 · K/32 == mma evaluated on every row:

# stmts M N K mma M/16·N/8·K/32 holds? weight bytes (rel. W) bias bytes
1 s452–s595 192 384 128 144 12·48·4 = 2304 no 0…11788 (+12288·tid.z) 49152…49328 (+192·tid.z)
2 s7089–s7112 96 32 32 24 6·4·1 = 24 yes 50112…50636 –
3 s7120–s7143 96 32 32 24 24 yes 53184…53708 –
4 s7151–s7174 96 32 32 24 24 yes 51136…51660 –
5 s7182–s7205 96 32 32 24 24 yes 54208…54732 –
6 s7213–s7236 96 32 32 24 24 yes 52160…52684 –
7 s7244–s7267 96 32 32 24 24 yes 55232…55756 –
8 s7439–s7486 64 96 32 48 4·12·1 = 48 yes none 56256…68044
9 s11330–s11377 192 96 96 48 12·12·3 = 432 no none –
10 s11547–s11558 16 96 32 12 1·12·1 = 12 yes 68544…71116 –
11 s11578–s11589 16 96 32 12 12 yes 90048…92620 –
12 s11609–s11620 16 96 32 12 12 yes 111552…114124 –
13 s11640–s11651 16 96 32 12 12 yes 133056…135628 –
14 s12288–s12299 48 96 96 12 3·12·3 = 108 no 136512…139084 139584…139632
15 s12557–s12568 16 96 32 12 12 yes 139648…142220 –
16 s12629–s12640 48 96 96 12 108 no 142912…145484 145984…146032
17 s12805–s12816 16 96 32 12 12 yes 146048…148620 –
18 s12877–s12888 48 96 96 12 108 no 149120…151692 152192…152240
19 s13053–s13064 16 96 32 12 12 yes 152256…154828 –
20 s13125–s13136 48 96 96 12 108 no 155328…157900 158400…158448
21 s13301–s13312 16 96 32 12 12 yes 158464…161036 –
22 s13373–s13384 48 96 96 12 108 no 161536…164108 164608…164656
23 s13549–s13560 16 96 32 12 12 yes 164672…167244 –
24 s13621–s13632 48 96 96 12 108 no 167744…170316 170816…170864
25 s13797–s13808 16 96 32 12 12 yes 170880…173452 –
26 s13869–s13880 48 96 96 12 108 no 173952…176524 177024…177072
27 s14045–s14056 16 96 32 12 12 yes 177088…179660 –
28 s14117–s14128 48 96 96 12 108 no 180160…182732 183232…183280
29 s14293–s14304 16 96 32 12 12 yes 183296…185868 –
30 s14365–s14376 48 96 96 12 108 no 186368…188940 189440…189488
31 s14541–s14552 16 96 32 12 12 yes 189504…192076 –
32 s14613–s14624 48 96 96 12 108 no 192576…195148 195648…195696
33 s14789–s14800 16 96 32 12 12 yes 195712…198284 –
34 s14861–s14872 48 96 96 12 108 no 198784…201356 201856…201904
35 s15037–s15048 16 96 32 12 12 yes 201920…204492 –
36 s15109–s15120 48 96 96 12 108 no 204992…207564 208064…208112
37 s15285–s15296 16 96 32 12 12 yes 208128…210700 –
38 s15505–s15510 48 48 96 6 3·6·3 = 54 no 211200…212236 212736…212752

Every phase whose M/16·N/8·K/32 does not equal mma is one the tool also marks grid = band, and every phase that holds is marked full. The mismatch factors are not constant (P1 ×16, P9 ×9, P14/…/P36 ×9, P38 ×9), so M, N and K cannot all be taken at face value for those rows. For P14 the reason is checkable and structural:

s12288 D=%r12342 A=%r12414 B=%r12293 C=bias@139584      (bo 136512)
s12290 D=%r12362 A=%r12434 B=%r12297 C=%r12342           (bo 137024)

— the same D accumulates products with B fragments from different 512-byte blocks and two different A quads. Under the settled weight formula (n = 16·tt + 8·(b>>3) + (L>>2), verified on enc0) P14's six blocks span n = 0…95, so N = 96 is supported; then the 3-mma chains must be summing over M or K, and neither is separately readable from the fragment structure. See §7/U2.

Which phases read no weight operand: P8 and P9. Both take B from registers the preceding epilogue packed (E7 packs phases 3…7's D for P8's A and B; E8 packs the probabilities and the transposed P7 for P9's A and B). Their bo is None for every mma.

Weight image extent as the tool measures it: the largest resolved offset is 212752 (s15504, 4 bytes wide) ⇒ 212756 bytes (1082 distinct slots). Phase 1's B and C bias carry the only tid.z terms (W + 12288·tid.z, W + 192·tid.z); with tid.z ∈ {0,1} (E38 rejects tid.z > 1 at s15561 setp.gt.u32 %p406, %r19503, 1) the extra 12288 bytes stay inside 212756. (INFERENCE — the tool drops the tid.z term, so the per-window base cannot be read off it directly.)

6. The plane arena (param_0 + 48) and the per-launch tables

  • param_0+8 is read 60 times in the prologue (s66…s451) as the input feature plane that builds phase 1's 12 A fragments.
  • param_0+24 is the patch-gather source (see §4); it is read, never written.
  • param_0+48 is the output plane arena: written by E37's 12 st.global.u32 (s15359…s15481), never read. The entry has exactly 12 st.global and no other global store.
  • The only other ld.param sites are the 4 in the prologue (§1.1) and the 5 in E38 ([%rd635+88/96/104/112/136], s15683/15759/15761/15582/15677) plus [%rd7+48] (s15333), [%rd7+24] (s2288) and the two param_0+40 re-reads (s15483/s15484).

There is no per-launch table generated into the plane arena by this kernel: the arena is only written (E37), and every table that looks like one (W + ((laneid<<2)&12) + {49920, 136128, 136320, 142720, …}) lives in the weight image, not the arena.

7. Uncharacterised

U1. E9's residual registers. The 24 registers that E9's per-column table adds into (%rs456/%rs459 and their siblings, s11473–s11544) have exactly one textual definition each, at s5693/s5694 (L8430/L8434) inside E1 — nothing between s5694 and s11473 writes them (grep -n '%rs456\b' returns L8430, L8913, L11184, L24297 only). So either the value is genuinely carried from E1's sum-of-squares tree, or the printed PTX reuses a register name across two disjoint live ranges without a rename. The two readings differ in what phase 10's C is (a table plus E1's partial sums, versus a table plus something unnameable), so I will not guess. Missing: a shared-memory/register dump at s11473, or the pre-rename SSA.

U2. The true M/N/K of the thirteen banded phases (P1, P9, P14, P16, …, P36, P38). For those rows M/16·N/8·K/32 ≠ mma (§5). nA counts distinct A register quads, which can be either distinct M-tiles or distinct K-slices of the same tile, and the accumulator chains mix distinct B operands, so the two cannot be separated from the fragment structure. Missing: a run with a weight-image dump, the _prep permutation, or the high-level layer definition.

U3. The identity of the 12288-byte score bias (P8's C). E7 loads it as 24 tiles × 512 bytes at W + 56256. Whether it is a relative position bias, a learned score bias, or a per-lane constant cannot be decided from the PTX: it is an opaque blob in a permuted weight image. Missing: the image's layout description or the model's parameter names.

U4. The identity of the per-column tables. W+49920…50096 (E1 gain), W+136128…136304 (E9), W+136320…136496 (E13 gain), W+142720…142896 (E14) are all read with the same W + ((laneid<<2)&12) + off pattern (12 words per lane). Their role is computable (E1/E13: multiply the inverse RMS; E9/E14: added to a residual before the next GEMM) but their names are not. The GEMM biases at +49152, +139584, +145984, +152192, +158400, +164608, +170816, +177024, +183232, +189440, +195648, +201856, +208064, +212736 are the same case.

U5. E1's patch-expand geometry. As in the sibling kernel, the 36 guarded st.shared are emitted as ~400 basic blocks with 413 labels and I could not reduce the lane→(row,col) map to a closed form. Established: 34 u32 + 2 v2.u16 patch stores, a second staging area of 6 v4.u32 stores at smem + tid.z·{2048,3072} + {0,512,1024,6144,6656,7168}, and the whole 9600-byte shared allocation. Missing: the source-level patchify expression or a shared-memory dump.

U6. Whether param_0+8 is an input plane or a scratch table. The prologue reads it 60 times with 4-byte elements and never writes it, and E1 does not read it (E1 reads param_0+24). So +8 and +24 are two different input buffers; which one holds the noisy colour plane and which the feature planes cannot be decided from the PTX. Missing: the host-side launch arguments.

U7. The tid.z strides. Phase 1's B and bias carry ×12288 and ×192 per-window strides and no later phase carries one. Whether the image is a concatenation of per-window sub-images or tid.z selects a batch element whose weights are interleaved cannot be decided here; consequently the "212756 bytes" of §5 is a lower bound.

U8. The logical meaning of E38's final expression. The tail computes out = acc·σ(v) + tex96·(1−σ(v)) + tex104 on the three channels gathered from the 3×3 colour stencil and the two integer-coordinate textures. The PTX gives the arithmetic exactly (§3/E38(g)) but not what model term it corresponds to. Missing: the high-level source or a reference implementation.

U9. Why every even phase shares one A operand. E13 emits two scaled copies of phase 13's D (the normed packs %r12414… at s12252 and %r18435… at s12158) and phases 14, 16, …, 36 all pack the second one. That makes the A operands of those twelve GEMMs bit-identical; whether that is the intended computation (one shared projection) or a consequence of a missed optimisation cannot be decided from the PTX. Missing: the high-level layer definition.

8. Quick acceptance index

requirement where
every epilogue named and ranged §2, §3 (E1…E38)
phase-1 epilogue's patch expand + norm §3/E1
the norm (E1, E13) §3/E1, §3/E13 — raw sum of squares + 2⁻¹³, no mean, no 1/N
the attention (score, softmax, P·V) §3/E7, E8, E9; §3/E9's table add
the activations §3/E14 (materialises %f861…%f864), E16, E18, …, E36
the patch expand §3/E1 (36 st.shared), §7/U5
the final epilogue, textures at +96/+104, surface §3/E38, §4
the texture reads at +96/+104 §3/E38(f), s15760 / s15762
the skip input at +24 §3/E1 (s2288), §4
the plane-arena writes §3/E37 (s15359…s15481), §4, §6
the phase table with M/N/K checked §5
weight image extent §5 (212756 bytes, 1082 slots)
which phases read no weight §5 (P8, P9)
uncharacterised §7