← Andrei Vogulkin Β· all writing

🏰 Tower Defense β€” model shootout

β†’ also: the agentic research shootout (LangGraph Γ— Atlas codebase)

One complete HTML5 game per configuration, identical 2-round prompts (full build β†’ review/fix/balance/polish), thinking disabled, temp 0.7 Β· top_p 0.8 Β· 26K tokens/round. NVIDIA DGX Spark GB10 (128 GB unified), 2026-08-28.

Model configuration Results
Config Model Engine & serving Memory tok/s R1β†’R2 Build time Game size Outcome
P
prod
Huihui-Qwen3.8-27B-abliterated
27B dense Β· Q4_K_M GGUF (17 GB)
llama.cpp DFlash build (PR #27342)
DFlash2 speculative decoding (n-max 7)
Sharp v22.3 template Β· 128K ctx
17 GB 46.6 β†’ 49.0 ~9 min 35.8 KB βœ… works
functional, plainest design
Play
E2 Qwen3.8-Flash-Next
125B MoE Β· 6B active Β· qwen4exp
Unsloth UD-Q3_K_XL (84 GB)
llama.cpp PR #27742 (unmerged)
51B n-gram/PLE β†’ CPU/mmap
-ot per_layer_token_embd=CPU
64K ctx Β· f16 KV
~89 GB 24.6 β†’ 20.0 ~32 min 56.1 KB βœ… works
most polished game: splash + lore, hotkeys, sound, damage text
Play
A Qwen3.8-Flash-Next (same)
Unsloth UD-Q4_K_XL (111 GB)
llama.cpp PR #27742
n-gram/PLE β†’ CPU/mmap
128K ctx Β· q8_0 KV
LLAMA_ATTN_ROT_DISABLE=1
~111 GB 23.4 β†’ 19.4
fix: 19.4
~31 + 16 min 50.2 KB βš οΈβ†’βœ… works
shipped with 1 fatal typo, then self-fixed (single-char diff) given the console error
Play

Round 2 β€” Atlas engine (2026-09-03/06) Β· performance & bug-fixing log ↓

Same prompts, same sampling, same hardware. Flash-Next served by Atlas (Rust/CUDA engine, RadixArk NVFP4 checkpoint, QSA sparse attention + PLE n-gram table) instead of llama.cpp β€” first on main@4df6e609, then on a branch carrying seven engine fixes found during the first run (assist/flashnext-longform-fixes).

ConfigModelEngine & servingMemorytok/s R1β†’R2Build timeGame sizeOutcome
ATLAS
main
Qwen3.8-Flash-Next
RadixArk NVFP4 (126 GB on disk)
Atlas 4df6e609
single-chunk prefill (20K), 48K ctx, watchdog off, no speculative
11 serve attempts to get one complete run
~100 GB 17.3 β†’ 16.3
fix: 16.6, 16.4
~17 + 27 min
+2 fix rounds (17 + 18)
47.8 KB ⚠️ dead β†’ loads
R1: 4 unclosed script blocks; R2: three rewrites, hit the 26K cap; loads clean after 2 fix rounds, play-through unverified
Play
ATLAS2
fixed engine
Qwen3.8-Flash-Next (same) Atlas branch 97aa9fa4 (7 fixes)
default 8K chunked prefill, 64K ctx, no env pins
one attempt, zero server errors
~100 GB 17.3 β†’ 16.7 ~9 + 14 min 38.2 KB ⚠️ inert
"BULWARK": syntax-clean, complete file β€” but defines everything and never boots (no newRun() call, no frame loop, no event listeners)
Open
ATLAS3
fixed engine + llama.cpp sampling
Qwen3.8-Flash-Next (same) Atlas branch cb6b1a59 (7 fixes, merged main)
default 8K chunked prefill, 64K ctx
request: presence_penalty 0 Β· top_k 40 Β· min_p 0.05 (Atlas's model preset otherwise applies presence 1.5)
~100 GB 17.0 β†’ 16.0
fix: 15.5
~19 + 20 min
+ 20 min fix
51.2 KB βœ… playable after 8 fix rounds (+1 reviewer token)
"Ironline": full game, RAF loop, 12 listeners; shipped with one typo, self-fixed byte-identical except that token. Then four logic bugs (intro state, path in cell units, HUD never written, 0/0 reward) each needed its own literal brief; every "extra" edit the model made on its own introduced a regression. Chain: fix2 Β· fix3 Β· fix4 Β· fix5 (plays, NaN gold) Β· fix6 Β· fix7 Β· fix8 (final; road no longer buildable β€” path-distance NaN from duplicate waypoints)
Play
ATLAS4
rollback rework Β· BF16 head + attention
Qwen3.8-Flash-Next (same NVFP4 experts) Atlas branch 7db64cca (PR #23 second batch)
BF16 lm_head (new default) + --attn-proj-dtype bf16
same request sampling as ATLAS3; watchdog off for the quality run
~102 GB 15.2 β†’ 14.0 ~22 + 27 min 57.7 KB ⚠️ one logic bug
"Bastion Defense": clean syntax, boots, road, placement, combat (36 shots / 5 kills in wave 2) all work; but killed and leaked enemies are never removed from the enemy list, so a wave never registers as cleared. Zero syntax errors vs ATLAS3's typo + 4 logic bugs β€” n=1, suggestive not conclusive. Speed cost of the BF16 projections β‰ˆ 10%.
With the watchdog on (same build, run "ATLAS4"): three rollback+re-steers and zero indexer desyncs (item 3 verified), but the loop detector false-positived on a coordinate array and cut a coherent file at 8.4K tokens.
Open
ATLAS5
A1 + A2 + A3 from the gap study
Qwen3.8-Flash-Next (same NVFP4 experts) Atlas branch 0b9ada13 (PR #23 third batch)
BF16 head + --attn-proj-dtype bf16 as ATLAS4
+ device-side QSA selection (ATLAS_QSA_DEVICE_TOPK=1), 4M-slot PLE row cache, event-scoped PLE pin release; watchdog off; same request sampling
~102 GB 14.7 β†’ 13.9 ~26 + 29 min 62.2 KB βœ… playable after 1 fix round
"IRONGATE": briefing, road, placement rules, three tower types, waves, kills and leaks, win/lose all work on first load; zero syntax errors. One bug: the stats panel (gold/lives/wave/score) was refreshed only on shop clicks, never from the frame loop β€” a 2-line fix round (fix1, exactly the two calls asked for, nothing else) made it live. Speed is unchanged vs ATLAS4 (14.7 vs 15.2 on a longer output): none of A1/A2/A3 moved decode β€” the PLE barrier is one layer, the tensor-core MoE arm is 3Γ— slower at M=1, and the QSA host round trips were not the bottleneck either. The gap is in the decode kernels themselves (MoE GEMV, GDN, hyper-connections).
Play
ATLAS6
profile-driven: NVFP4 GDN + x4 hyper-connection kernel
Qwen3.8-Flash-Next (same NVFP4 experts) Atlas branch e7b6af2f (PR #23 fourth batch)
GDN projections NVFP4 (ATLAS_QWEN4EXP_BF16_GDN=0), BF16 head, NVFP4 attention, hc_pre_finish_x4, device top-k, 4M PLE cache; 48K ctx; watchdog off; same request sampling
~108 GB 21.6 β†’ 20.3 ~16 + 17 min 52.9 KB βœ… playable after 3 fix rounds
"IRONWORKS Β· TD": fastest Atlas run yet (+47% vs ATLAS5), file closed and syntax-clean, but shipped with two defects: the reset code called a helper by the wrong name (buildStatic vs bakeStatic) so nothing rendered (fix1 echoed the file; fix2 made the one-token rename), then the update loop died on an undefined lv in the turret branch (fix3: the two-line binding asked for). Play-through: four waves cleared, kills, gold and score all correct, 20/20 lives. Quality gate for NVFP4 GDN: two defects in one sample versus one for ATLAS5; not conclusive either way.
Play
Raw round data β€” Atlas runs
ConfigRoundTokensWall timetok/sFinishCode emitted
ATLASR1 build18,02717 m 19 s17.3stop47,611 ch (unterminated; 4/6 scripts unclosed)
ATLASR2 fix/polish26,62427 m 18 s16.3length35,208 ch (best of 3 rewrites; broken)
ATLASFix 1 (console error)16,68716 m 48 s16.6stop45,479 ch βœ“ loads
ATLASFix 2 (playtest bugs)17,47517 m 46 s16.4stop47,824 ch βœ“ loads
ATLAS2R1 build9,6129 m 16 s17.3stop26,675 ch βœ“ closed
ATLAS2R2 fix/polish13,86613 m 51 s16.7stop38,163 ch βœ“ closed, never boots
ATLAS3R1 build18,95418 m 34 s17.0stop51,198 ch βœ“ closed
ATLAS3R2 fix/polish18,95319 m 44 s16.0stop51,220 ch βœ“ closed (1 fatal typo)
ATLAS3R3 self-fix18,98020 m 22 s15.5stop51,222 ch βœ“ (1-token diff; intro-state bug remains)
ATLAS3fix2 (runtime brief: intro state, road)19,01419 m 51 s16.0stop51,218 ch β€” state fix correct, but a stray } broke the script
ATLAS3fix3 (console error)30,33435 m 55 s14.1stop51,218 ch β€” one statement changed; 11K tokens of extra output
ATLAS3fix4 (runtime brief: overlay, path units)19,07320 m 24 s15.6stop51,267 ch β€” fixed the two lines named, ignored the other path consumers
ATLAS3fix5 (literal brief: pixels + HUD)19,31820 m 32 s15.7stop51,495 ch β€” playable; kill reward 0/0 β†’ NaN gold
ATLAS3fix6 (runtime brief: NaN)19,25920 m 11 s15.9stop51,552 ch β€” NaN fixed; an unasked edit reintroduced the original typo
ATLAS4 (watchdog on)R1 build8,4019 m 13 s15.2watchdog cap21,201 ch β€” cut by 3 false-positive rollbacks on a coordinate array
ATLAS4 (watchdog off)R1 build20,21722 m 08 s15.2stop52,954 ch βœ“ closed
ATLAS4 (watchdog off)R2 fix/polish22,43226 m 39 s14.0stop57,685 ch βœ“ closed, 0 syntax errors, 1 logic bug
ATLAS5R1 build23,22726 m 24 s14.7stop61,907 ch βœ“ closed
ATLAS5R2 fix/polish23,71128 m 30 s13.9stop62,179 ch βœ“ closed, 0 errors, playable
ATLAS6R1 build20,62115 m 57 s21.6stop52,597 ch βœ“ closed
ATLAS6R2 fix/polish20,69716 m 59 s20.3stop52,854 ch βœ“ closed; calls undefined buildStatic()
ATLAS6fix1 (console error)20,71917 m 35 s19.6stopfile echoed unchanged
ATLAS6fix2 (literal brief: rename)20,39916 m 39 s20.4stopexactly the one-token rename; board renders; update loop then throws on lv
ATLAS6fix3 (literal brief: bind lv)20,51617 m 12 s19.9stop52,880 ch β€” exactly the two bindings; playable (4 waves cleared)
ATLAS5fix1 (runtime brief: stats panel)23,58129 m 28 s13.3stop62,223 ch β€” exactly the two calls requested; panel verified live
ATLAS3fix7 (console error)19,04620 m 10 s15.7stop51,554 ch β€” single-token diff; fully playable (3 waves cleared, 20/20 lives, HUD + log live)
ATLAS3fix8a (runtime brief: road placement)16,48018 m 17 s15.0stopno code block β€” explanation, then EOS mid-file
ATLAS3fix8 (tighter brief, rerun)19,01222 m 31 s14.1stop51,632 ch β€” both requested fixes correct; an unasked edit broke the overlay grid again (reviewer reverted that one token to ship the final page)
Engine defects found on main, and what the branch fixes
D-1  loop-watchdog rollback desynced the QSA indexer ("pos 2864 but 2932 ingested") β†’ 500
D-2  speculative verify + QSA active at 2051 tokens β†’ error laundered into finish_reason="stop"
D-3  prompts > 8192 tokens refused twice (PLE cap off by one vs the 8193 chunk; 65536-slot n-gram row cache)
D-4  continuation prefill chunks faulted CUDA_ERROR_MISALIGNED_ADDRESS; engine lingered with a dead context
D-5  ATLAS_QSA_MAX_TOKENS default 32768 aborted decode mid-stream

Branch assist/flashnext-longform-fixes (7 commits on 4df6e609):
  ple: derive the n-gram row cache from the serve config; clamp ATLAS_PLE_MAX_TOKENS up          (D-3)
  qsa: size the indexer from --max-seq-len instead of a fixed 32768                               (D-5)
  scheduler: report engine aborts as finish_reason "error"; log cuBLASLt fallbacks                (D-2b)
  scheduler: stop the loop once a GPU fault is latched                                             (D-4 aftermath)
  rollback: rewind QSA/PLE aux state with the SSM state, or decline                                (D-1; restore path works, cursor 1 token behind β€” open)
  scheduler: latch MTP off before a sequence can cross the QSA inert bound                        (D-2a)
  mhc: align the prefill GEMM scratch sub-buffers; odd slabs faulted the GEMM                     (D-4 root cause)

D-4 root cause: hc_pre_gemm packed its BF16 scratch as raw byte products; inj_pre is lay*8 bytes, so an ODD
prefill chunk (1,117 tokens, or the default 8,193) put up_wt 8 bytes off a 16-byte boundary and the tensor-core
GEMM faulted. Found with CUDA_LAUNCH_BLOCKING=1 (compute-sanitizer was ~100x too slow to reach the chunk).
Verified: the failing 13,397-token prompt prefills at 4,096 and 8,192 chunking; ATLAS2 ran end to end with defaults.

Sampling was the quality lever. Atlas fills unspecified request fields from the model's [sampling.non_thinking] preset β€” top_k 20, min_p 0 and presence_penalty 1.5 (the vendor card's chat value) β€” while llama.cpp used its own defaults (top_k 40, min_p 0.05, no penalty). A flat βˆ’1.5 on every already-emitted token is poison for 20K-token code (closing tokens, repeated keywords, boilerplate all get suppressed): the dead/inert games of ATLAS and ATLAS2. With the request pinned to llama.cpp's values (ATLAS3) the model wrote a complete, wired 51 KB game and, like config A in August, self-repaired its one typo with a single-token diff. What remains is an ordinary logic bug.

Speed did not change with the fixes (16–17 tok/s single-stream, ~430 tok/s prefill): they remove failure modes, not cost. Atlas remains ~3x slower than prod and ~20–30% slower than llama.cpp on this model; its quality-of-output on the task was worse in both runs (dead or inert games vs. llama.cpp's playable ones), which is a sampling/serving-stack difference worth its own investigation.

Performance & bug fixing β€” decision log (ATLAS β†’ ATLAS6)

Every Atlas round in order: what was decided, what was changed (commits and files on the fork branch andrew-vogulkin/atlas:assist/flashnext-longform-fixes, PR Atlas-Inf/atlas#23), and what it did to speed and output. Speeds are single-stream decode on the same NVIDIA GB10; llama.cpp serves the same model there at 24–25 tok/s.

RoundEngine headDecisionR1 / R2 tok/sGame
ATLASmain @ 4df6e609serve as-is; find out why it fails17.3 (11 attempts)fix2 needed to boot
ATLAS2cb6b1a59fix the five engine defects, run on defaults17.3 β†’ 16.7inert
ATLAS3cb6b1a59 + request samplingequalise sampling with llama.cpp17.0 β†’ 16.0playable after 8 fix rounds
ATLAS47db64ccaBF16 head + attention for quality; rollback rework15.2 β†’ 14.01 logic bug
ATLAS50b9ada13execute the gap-study items A1–A314.7 β†’ 13.9playable after 1 fix round
ATLAS6e7b6af2fprofile with nsys, act on the measured top two stages21.6 β†’ 20.3playable after 3 fix rounds

ATLAS β€” 2026-09-03 Β· main @ 4df6e609

Serve the RadixArk NVFP4 checkpoint on stock Atlas2026-09-03R1 17.3 tok/s Β· 11 serve attempts
Decision
Run the same two-round TD benchmark llama.cpp had run in August, on Atlas main, and record every failure rather than tune around it.
What happened
Five distinct engine defects over eleven serve attempts: watchdog rollback desyncing the QSA indexer (HTTP 500), MTP verify with QSA active ending the response as a clean stop (silent truncation), prompts over 8,192 tokens refused twice at the PLE layer (cap off by one, then a 65,536-slot row cache), continuation prefill chunks faulting with CUDA_ERROR_MISALIGNED_ADDRESS and a zombie process holding 100 GB, and ATLAS_QSA_MAX_TOKENS defaulting to 32,768 regardless of context.
Workarounds used
ATLAS_PLE_MAX_TOKENS, ATLAS_QSA_MAX_TOKENS, ATLAS_PLE_CACHE_SLOTS env pins, single-chunk prefill (--max-prefill-tokens 20480), --content-loop-watchdog false, no speculative decoding.
Output
Game "IRONGATE" needed a second fix round to boot. Speed 17.3 tok/s single stream, ~300 tok/s prefill.
Files
no engine changes; scripts ~/driver_td2*.sh, ~/tdbench_r2.py, ~/tdfix3.py on the bench machine; report bench report.

ATLAS2 β€” 2026-09-04 Β· fixed engine, default settings

Fix the five defects on a branch, then run with no env pins2026-09-04R1 17.3 β†’ R2 16.7 tok/s
Decision
Two independent investigations (prod model and config A) produced a seven-item plan; execute all seven with the config-A executor on one branch, then re-run the benchmark on defaults.
Commits
  • e623e010 PLE row cache derived from max_batch_tokens Γ— ngram_heads, env stays an override; cap clamped up with a warning β€” crates/spark-model/src/weight_loader/qwen4_exp/ple.rs, qwen4_exp.rs
  • 561be215 QSA indexer capacity from --max-seq-len (ModelConfig.max_seq_len) β€” crates/atlas-core/src/config.rs, config/factory.rs, spark-server/src/main_modules/serve_load.rs, spark-model/src/layers/qsa.rs, attach.rs
  • 16af4d33 engine errors reported as finish_reason="error" (500 for blocking clients) instead of stop; cuBLASLt failure logged with M/N/K β€” scheduler/types.rs, lifecycle.rs, ir/response.rs, anthropic/translate.rs, qwen3_ssm/trait_prefill_proj.rs
  • 35028544 scheduler breaks on a latched CUDA fault instead of holding 100 GB and 500-ing β€” scheduler/mod.rs
  • f4e81291 watchdog rollback saves/restores the QSA/PLE aux state with the SSM snapshot, declines when missing β€” traits/model.rs, model/types.rs, impl_a1.rs, model/trait_impl/mod.rs, scheduler/rollback.rs
  • 86bdf303 MTP latched off before a sequence crosses the QSA inert bound β€” scheduler/mtp_gate/qsa_latch.rs, scheduler/mod.rs, multi_seq/mod.rs
  • 97aa9fa4 hyper-connection prefill scratch laid out on 256-byte boundaries (odd slabs faulted the GEMM) β€” spark-runtime/src/buffers/sizes.rs, buffers.rs, ops/hyper_connection_lowrank.rs
  • d32605b2 merge of upstream main; cb6b1a59 test fix
Impact
Zero server errors across the run on default chunking with no env pins. Speed unchanged (the fixes remove failure modes, not cost). The game "BULWARK" was inert (no boot loop, no listeners): the quality problem was not the engine.

ATLAS3 β€” 2026-09-04 Β· sampling equalised

Pin the request to llama.cpp's effective sampling2026-09-04R1 17.0 β†’ R2 16.0 tok/s
Decision
Found in spark-server/src/api/chat/sampling_setup.rs: Atlas fills unset request fields from the model card's [sampling.non_thinking] preset, including presence_penalty 1.5. Over a 20K-token file that suppresses every token already emitted. Re-run with presence 0, top_k 40, min_p 0.05 on the same engine.
Impact
A complete, wired 51 KB game ("Ironline") with one typo; the identical engine had produced dead/inert files twice. Then eight fix rounds: the model repairs exactly what a brief names, and every unrequested edit it made introduced a regression (fix2, fix6, fix8 each re-broke one token; one round emitted EOS mid-file; one echoed the file). Playable after fix7; a road-placement bug (path distance NaN from duplicate waypoints) fixed in fix8 with one reviewer token reverted. Per-round data on the bench page.
Commit
953820a0 [behavior].use_sampling_preset_penalties_for_core (default true, Flash-Next opts out) β€” crates/atlas-kernels/build_parse_behavior.rs, build.rs, build_codegen.rs, src/lib.rs, spark-server/src/api/chat/sampling_setup.rs (+ tests), kernels/gb10/qwen3.8-flash-next/MODEL.toml

ATLAS4 β€” 2026-09-04 Β· precision and rollback

BF16 head and attention for quality; make the watchdog rollback correct2026-09-04R1 15.2 β†’ R2 14.0 tok/s (watchdog off)
Decision
Attribute the remaining quality gap to precision: the checkpoint ships attention and lm_head in BF16 and Atlas requantised both to NVFP4 at load. Keep the head BF16 by default and add a flag for BF16 attention projections. Finish the rollback fix (the earlier item left a one-token lag).
Commits
  • 61f1993b BF16 lm_head default for qwen4_exp β€” crates/atlas-core/src/config/methods.rs
  • c120f479 --attn-proj-dtype bf16 β€” atlas-core/src/config.rs, spark-server/src/cli/{serve_args,validate,flag_values}.rs, main_modules/serve_load.rs, spark-model/src/weight_loader/qwen35/load_layers/attention_arms.rs
  • 6c5b61c3 + 7db64cca rollback rewinds by fed tokens (the two watchdog sites sit on different sides of the step's push) and drops the token sampled from the discarded context β€” scheduler/rollback.rs, rollback_tests.rs, decode_logits_content.rs, decode_logits_step.rs
Impact
Rollback verified on the GPU: five rollback + re-steer events, zero indexer desyncs. Two side findings: the content-loop watchdog false-positives on code (it cut a coherent file at a coordinate array), so it stays off for code; and each BF16 choice costs 5–6% decode (both β‰ˆ 12%). Game: 57.7 KB, zero syntax errors, one logic bug (killed enemies never removed). The speed drop from ATLAS3 is the precision trade, measured later per half: BF16/BF16 13.8 Β· BF16/NVFP4 15.0 Β· NVFP4/BF16 14.7 Β· NVFP4/NVFP4 15.6 tok/s at 3.9K context.

ATLAS5 β€” 2026-09-04 Β· gap study executed

Compare the two engines from source, execute the three cheapest items2026-09-04R1 14.7 β†’ R2 13.9 tok/s
Decision
Two investigations by the prod model (Atlas-only decode anatomy, then Atlas vs llama.cpp side by side, gap study) proposed six items. Measured first: CUDA graphs are worth β‰ˆ3% to llama.cpp (GGML_CUDA_DISABLE_GRAPHS=1: 24.8/25.1 β†’ 24.4/24.1), so capture was dropped to last. Execute A3, A2, A1 with the executor; verify each contribution.
Commits
  • 69cb6625 A3: event-scoped PLE pin release + pinned slot staging (the PLE is one layer, so one barrier per token) β€” spark-runtime/src/gpu.rs, cuda_backend/gpu_impl.rs, spark-model/src/layers/ple/{aux_state,layer,verify}.rs β€” neutral
  • ad7203f5 A2: ATLAS_MOE_DECODE_ARM=grouped selector β€” spark-model/src/layers/moe/forward.rs, MODEL.toml β€” grouped tensor-core arm at M=1 is 3Γ— slower (5.3 vs 15.0 tok/s); scalar stays
  • 0b9ada13 A1: device-side QSA block selection qsa_select_topk behind ATLAS_QSA_DEVICE_TOPK=1, parity mode ATLAS_QSA_TOPK_VERIFY=1 β€” kernels/gb10/qwen3.8-flash-next/nvfp4/qsa_indexer.cu, spark-model/src/layers/ops/qsa.rs, layers/qsa.rs, new layers/qsa_decode_select.rs β€” parity clean; +3–5% only once the sparse selection is active (past 2,051 tokens)
Impact
Decode unchanged on the full benchmark: host synchronisation, PLE placement and graph capture together are worth a few percent. Game "IRONGATE" (62 KB) played out of the box except a stats panel refreshed only on shop clicks; one two-line fix round. Conclusion recorded: the gap is in the decode kernels.

ATLAS6 β€” 2026-09-06 Β· measured, not reasoned

Profile one decode window with nsys, act on the two largest stages2026-09-06R1 21.6 β†’ R2 20.3 tok/s
Decision
Stop estimating. 40 s of pure decode under nsys (642 tokens, ~69 ms each): dense_gemv_bf16 73 launches = 24 ms (42%) β€” 72 of them the 36 GDN layers' in/out projections, which the loader keeps in BF16 (4.15 GB/token, bandwidth-bound); hyper-connection stage 3 hc_pre_finish 97 Γ— 121 Β΅s = 11.7 ms (27% with its siblings); MoE β‰ˆ 8.3 ms at ~77% of achievable bandwidth (already near its floor, so the proposed GEMV rewrite was dropped). The prod model's own reading of the same profile misattributed the MoE term and was overruled by the numbers. Profile: reports/atlas-flashnext-decode-profile-2026-09-06.md.
Changes
  • Flag ATLAS_QWEN4EXP_BF16_GDN=0: requantise the GDN projections to NVFP4 through the loader path that already existed (its comment, written at a 2.2 tok/s era, said GDN bandwidth "is not what bounds decode") β€” 15.0 β†’ 18.3 tok/s; costs ~6 GB (context 64K β†’ 48K at 0.95 utilisation)
  • e7b6af2f hc_pre_finish_x4: one warp per hyper-connection stream, lanes over 32 output dims (4Γ— the threads), identical FP32 accumulation order under --fmad=false β€” greedy output hash identical to the previous binary; 18.3 β†’ 20.3 tok/s β€” kernels/gb10/qwen3.8-flash-next/nvfp4/hyper_connection.cu, spark-model/src/layers/ops/hyper_connection_lowrank.rs, hyper_connection_lowrank_gemm.rs (ATLAS_HC_FIN_X4=0 reverts)
  • 3fff25af hc_pre_finish rank loop unrolled Γ—8 β€” neutral (the kernel was thread-starved, not load-starved; kept as documentation)
  • 22321152 qsa_select_topk scores staged in shared memory β€” neutral on tok/s, bit-exact
Measured, rejected
ATLAS_GDN_FUSED_NORM=1 (recurrence + gated norm fused): no speed change and the greedy output differs. ATLAS_D2H_SPIN_SYNC, ATLAS_PLE_FAULT_THREADS=128: neutral. FP8 / NVFP4 lm_head: +1 / +1.5 tok/s on top, at the documented argmax-flip risk; not adopted.
Impact
Full benchmark 21.6 / 20.3 tok/s (+47% over ATLAS5, ATLAS3 was 17.0 on NVFP4 head and attention). Game "IRONWORKS Β· TD" shipped with two defects (a helper called by the wrong name, an unbound level variable in the turret loop); a console-error round echoed the file, two literal-brief rounds fixed exactly what was named; then four waves cleared cleanly. Quality gate for the NVFP4 GDN requantisation: two defects in one sample versus one for ATLAS5 β€” not conclusive.
Left on the table
BF16 lm_head β‰ˆ 6 ms/token (precision decision); hc_pre_down and the other hyper-connection launches β‰ˆ 5 ms; conv1d launched separately from the GDN recurrence β‰ˆ 1 ms (the existing fused kernels are not bit-exact); ~1,000 launches/token of gaps. llama.cpp: 24–25 tok/s.

Method notes that shaped the decisions

Every engine change is proven bit-exact by a greedy 160-token generation on the previous binary and the new build (hash compare) before its speed is quoted; the one lever that failed that test was rejected. The prod model ran the investigations and the plumbing-shaped executor items (A3, A2) from anchored briefs; it produced no change for kernel items (A1, the HC kernel), which were written by the reviewer and stated as such. Game fix rounds follow the August protocol: console error first, then a runtime brief naming the exact change; regressions from unrequested edits are logged rather than hidden. Speeds at "3.9K context" are 1,200-token generations after a 3,882-token prompt so the sparse attention path is active; benchmark speeds are the two 20K-token rounds.

Prompts

Round 1 β€” build prompt (identical for all configs)
Build a complete, polished single-file HTML5 tower-defense game (index.html: all CSS and JS inline, canvas-based, no external assets or libraries). Requirements:
- 20x12 grid map with a fixed winding enemy path (visually distinct road) and buildable cells
- 3 tower types: Gun (fast, single-target), Cannon (slow, splash AoE), Frost (slows enemies). Each has cost, range, and 2 upgrade levels (better damage/range/fire-rate). Towers can be sold for 70% of invested cost.
- Enemies: normal, fast, tank; a boss every 5th wave. Health bars above enemies. HP/speed/count scale each wave.
- Economy: start with 100 gold and 20 lives; kill bounties; wave-completion bonus; a leaked enemy costs 1 life (boss costs 5).
- UI: top bar (gold, lives, wave, score); tower shop panel with prices; click a shop tower then click the map to place (show range preview and valid/invalid cell highlight); click a placed tower to select it and show upgrade/sell panel; Start Wave button plus auto-start toggle; game speed x1/x2 button.
- Targeting: enemies closest to the exit first. Render projectiles. Show splash radius on cannon impact and a frost visual on slowed enemies.
- Win after beating wave 20; lose at 0 lives. Proper end screens with a Restart button.
- Polish: hover states, particle effects on kills, hit flashes, clean modern dark theme.
Output the COMPLETE index.html in one ```html code block. No placeholders, no TODOs β€” fully working code only.
Round 2 β€” review / fix / balance / polish prompt
Below is your current index.html for the tower-defense game. 1) Review it for bugs: logic errors, undefined variables, broken event handlers, targeting/pathing mistakes, economy bugs, rendering issues β€” fix them all. 2) Ensure every item of the original spec is implemented: 3 tower types with 2 upgrade levels + sell; 4 enemy types incl. boss every 5th wave; wave scaling; gold/lives/wave/score UI; place/select/upgrade/sell interactions with range preview; start-wave + auto-start + speed toggle; closest-to-exit targeting; projectiles + splash + frost visuals; win at wave 20 / lose at 0 lives with restart; particles and polish. 3) Balance: tune costs, bounties and wave scaling so waves 1-20 are beatable but challenging. 4) Polish: particles, hit flashes, nicer end screens. If the file below is cut off mid-code, complete it properly. Output the COMPLETE final index.html in one ```html code block.

Current file:
```html
{FILE}
```
Config A only β€” self-fix round (given its own console error)
Your single-file HTML5 tower-defense game below fails to load. The browser console shows this error:

    Uncaught SyntaxError: Unexpected token ')'

Find and fix the error. Also scan the whole file for any other syntax mistakes (unbalanced brackets/parentheses, broken statements) that would prevent it from running. Do not redesign anything β€” make the minimal corrections needed so the game runs. Output the COMPLETE fixed index.html in one ```html code block.

Current file:
```html
{FILE}
```

Raw round data

ConfigRoundTokensWall timetok/sFinishCode emitted
PR1 build12,3654 m 25 s46.6stop35,591 ch βœ“
PR2 fix/polish12,5504 m 16 s49.0stop35,762 ch βœ“
E2R1 build21,14614 m 21 s24.6stop55,956 ch βœ“
E2R2 fix/polish21,50417 m 55 s20.0stop56,057 ch βœ“
AR1 build18,60813 m 16 s23.4stop48,934 ch βœ“
AR2 fix/polish19,19816 m 31 s19.4stop50,226 ch βœ“ (fatal typo)
AR3 self-fix18,90816 m 13 s19.4stop50,225 ch βœ“ (1-char diff)

Prompt processing (prefill)

ConfigShort prompt (84 tok)Long prompt (~13–21K tok, file embedded)Source
P~180–340 tok/s720.6 tok/s (12,750 tok in 17.7 s)server log of the actual R2 round
E2123.7 tok/s475.0 tok/s (20,983 tok in 44.2 s)dedicated probe, same file-embedded prompt
A118.7 tok/s465.7 tok/s (20,983 tok in 45.1 s)dedicated probe, same file-embedded prompt

Prefill = server-side prompt eval time from llama.cpp. Short prompts are dominated by fixed overhead on every config; on long prompts prod processes ~1.5Γ— faster than Flash-Next. In practice: feeding a ~50 KB file back for review costs ~18 s of prefill on prod vs ~45 s on Flash-Next, before any generation starts.

Method & pipeline

How this bench was run
Sampling: temperature 0.7, top_p 0.8, max_tokens 26,624 per round, thinking disabled
(chat_template_kwargs: enable_thinking=false β€” mandatory: with thinking on, models burned
the entire budget reasoning; prod produced 52K chars of thought and zero code).

Protocol: each config got the same Round-1 build prompt, then Round 2 received its own
full file back for review/fix/balance/polish. The harness extracted the single ```html
block from each response; the last complete file became the published game.

Config A shipped with `flashColor='#ff4d9d');` β€” one stray parenthesis that killed the
whole game on load. Given only its file plus the console error, it regenerated all 50 KB
byte-identical except that character. Verified with node --check before republishing.

Serving: prod runs live on spark:8000; Flash-Next configs run on :8001 with prod stopped
(128 GB unified memory is one pool β€” CPU offload frees nothing while another model is
resident). Publishing: spark β†’ orange ~/sites/bench β†’ orange Caddy (handle_path /bench*)
β†’ wg umbilical β†’ edge Caddy (TLS) β†’ you.

Speed note: prod got FASTER on this task than on short prompts (49 vs 38.6 tok/s) β€”
long uninterrupted code generation is ideal for DFlash2 speculative decoding. Flash-Next
slowed as context grew (24.6 β†’ 19.4 tok/s).

Spec: 20Γ—12 grid, 3 tower types with upgrades/sell, 4 enemy types incl. bosses every 5th wave, economy, wave scaling, win at wave 20. Both Flash-Next runs independently branded their game β€œIRONHOLD” and out-designed prod; prod built a working game 3.5Γ— faster. Verdict: prod for speed and agent duty, Flash-Next (E2) when craft beats latency.