agent-lessons

Verification, tests and gates

201 lessons. Most-repeated first.

• (mise, 2026-11) Never kill browsers with Get-Process headless_shell

  Stop-Process: other sessions share the machine and their Playwright runs die too (and so did my own detached checker, which then hung silently). Kill only your own tree: find children by ParentProcessId of your checker PID. Make long checkers print a flushed progress line per phase so a hang is visible.

• Voice in PWAs: never rely on Web Speech alone (missing in Firefox, silent in iPhone home-screen apps). Keep a recorder + server STT fallback, enforce limits with timers (not audio callbacks), and test with a real WAV through Chromium’s fake mic (one fresh browser per scenario). Playwright’s Windows WebKit has no getUserMedia or AudioContext, so stub them to emulate iOS.
• D report must survive one slow site: a 180 s audit timeout used to abort the whole report (batam-uncle-says, 2026-11-07); it now records ‘audit timed out’ and moves on. Sessions run in parallel: re-check paths (the skill folder was renamed refine -> hone mid-run) and re-run gates right before publishing.
• A project that moved needs its CTA links changed in state/content/<slug>.json and the stored state/data.json entry (ingest refuses rebuilt pages). Declare the old->new pairs in MOVED_LINKS in tools/check.py so G3 accepts the move.
• When a council finding claims a race or crash bug, prove it with a real reproduction (a real kill, a real NFS mount, a real protected branch) before and after the fix; simulated tests found none of the bugs the real ones did.
• Local Playwright checkers on Windows: use ThreadingTCPServer and end with os._exit() after writing results; a single-threaded server plus browser.close()/server.shutdown() hung for minutes after the work was done.
• Long council/test runs: launch with creationflags DETACHED_PROCESS so a Python-tool timeout or restart does not kill them; poll the output files.
• Local and live audits gave identical counts for Cairn, so baseline/after both on the local checkout is a fair comparison.
• ScopeGuard now lives in juw12/steward/apps/playground; Atomize’s app code isn’t public (README-stated stack only).
• Exact-path checks only: crewcup/index.html ends with index.html, so suffix matching flags the gallery falsely.
• juw12/projects is a monorepo: the checkout is sparse; git sparse-checkout add <path> happens automatically.
• check.py output redirected by PowerShell (>) is UTF-16; landing.py check avoids that by capturing it.
• Gates can miss lost content: diff against live before publishing. Verify deploys with ?cb=<timestamp>.
• The contrast probe skips text over background images; check those by eye in the screenshots.
• Some apps’ live URL carries ?demo=1; the local audit reuses that query string.
• GitHub Pages takes ~1 min to deploy; always verify live with ?cb=<timestamp>.
• Playwright test/shot scripts must not call browser.close() and must print with flush=True; hung runs left 20 headless_shell processes that pinned the CPU and made every later run stall. Kill only processes whose command line contains ms-playwright or the run’s own script names. When CPU stays at 100% from other apps, stop on budget and keep the last judged best instead of judging a half-rendered candidate.
• When a hone landing page still shows emoji or README artifacts after a helper fix, check which render path that block actually uses; the checklist block renders d[checklist] verbatim, not via first_sentence. Fix at the render site, verify the LIVE page with slop_check.py (target 0 blockers), then publish via landing.py publish –only SLUG –push (site clone must be clean, commits p/ only). (correction from the human)
• Piping a value into gh secret set from PowerShell appends CR/LF, which breaks base64 -d in CI (and would corrupt a password). Set secrets with gh secret set NAME --body <value> from Python, and decode with tr -d '\r\n ' first. Android immersive mode shows a one-time ‘Viewing full screen’ system sheet that swallows the first tap in emulator smoke tests.
• A canvas game whose rAF loop calls requestAnimationFrame(frame) at the END of frame dies silently on the first exception (sim “stops” with no pageerror after the first one). Schedule rAF first and wrap the tick in try/catch + console.error, and make the checker fail on console errors. Found via a “time stopped advancing” probe, not via pageerror.
• Auto-play bots must exercise every camera/view variant; the third-person walk path had an undefined target (task stored ticket, spot lookup read slot) that the first-person path never touched. Delayed UI timers (setTimeout showDay) must check they still belong to the current session.
• Global single-letter shortcuts collide with view-local keys (Map L = local map) - let the view preventDefault and check defaultPrevented; long mono chips need max-width+ellipsis at 320 px (correction from the human)
• A panel that skips re-render while it holds focus never updates after an embedded checkout (focus stays in the payment iframe). Force a redraw when the state the user paid for flips.
• A hand-built Design winner lives in landing/state/custom/<slug>/. build copies it, and the page needs <html class="dna custom"> or publish refuses it as an unbuilt seed.
• Gates caught i18n key collision + settings overwrite; real hardware still pending (OPPO not on ADB) (correction from the human)
• No need for hand-off tasks (verify mails, codes, gh login) - work around them instead of asking (correction from the human)
• Wants full auto, no review; human-check codes must still be typed by him (correction from the human)
• Cairn voice-log-3: (1) Hugging Face is blocked here; Vosk via jsDelivr (vosk-browser) plus the ccoreilly model host works for offline speech. (2) Feature modules that share generic class names collide (.log-row broke the mood row); prefix per module (.log-mood-row). (3) Date-dependent demo data passed on some days and failed on others; pin the clock in tests (page.clock.install) and assert the same brain on several dates. (4) Vector app-0.0.145 has no git/gh on PATH: use the full paths and set HTTPS_PROXY=http:// for gh. (5) landing data.json is indent-1 JSON with CRLF; round-trip and compare before editing, and keep the edit additions-only. (6) A parallel session merged its own fix for the same bug first; take theirs in the conflict and keep your regression test. (7) Background work: Start-Job dies when the shell tool returns and a cmd /c line with nested quotes silently never ran; write a .bat that redirects its own output and Start-Process it. (8) landing check.py rendered gates hang on page.goto 'load' because Chromium cannot fetch fonts.googleapis.com even with the proxy; run a scratch copy that page.route()s the font hosts to abort (same gates, fallback fonts). (9) A portfolio card copy edit needs no clone: GET scripts/apps.json via the contents API, replace the one line in the decoded bytes (CRLF kept), PUT with the sha, then trigger one sync-cards run.
• Two CI traps. (1) GitHub retired the macos-14 runner on 2026-11-02; jobs targeting it are cancelled in about 2 s with zero steps, and the only clue is a check-run annotation (“The configuration ‘macos-14-arm64-us-default’ is not supported”). Read gh api repos/O/R/check-runs/ID/annotations before guessing, and move to macos-15 / macos-15-intel. (2) PyObjC class names are process-global: two NSObject subclasses both named _Ticker in different modules crash the built app at import (“X is overriding existing Objective-C class”). Windows unit tests never import AppKit, so keep an AST test that checks ObjC class names are unique across the package.
• Check every capture before judging. Three rounds were lost to bad evidence, not bad UI: a run that had not started yet (“0/20 ready” shown as the finished state), a raw [WinError 32] toast from two runs writing the same project file, and a success toast covering the phone legend. Fixes: the app sets data-run=done when a run finishes and the audit waits on it (app_audit.py --ready-js); the audit retries goto with load when networkidle starves on a busy machine; the server serialises writes and retries PermissionError; never run probes while an audit runs.
• Headless checks of three.js scenes need --use-angle=d3d11 --ignore-gpu-blocklist --enable-unsafe-swiftshader, a data-ready flag set after the first frame, and a world.info() / tableOnScreen(n) test hook so the test can click real 3D objects. Lamps and string lights hung between the camera and the tables read as clutter; keep them on walls and edges. Floating labels at head height collide with seated avatars; put table tags just above the table top and hide the tag of the user’s own table. Stack chat bubbles (newest lowest, max 3).
• This PC has two Playwright installs for Python 3.13 (user-site Roaming 1.55 -> webkit-2203, and system site-packages expecting webkit-2287). A DETACHED_PROCESS launch picked the system one and skipped WebKit; launching via PowerShell Start-Process used the user-site one. Check playwright.__file__ in the actual test process before trusting “webkit not available”. Flaky single tap timeouts that move to a different control each run are load, not bugs; one real bug (mute) surfaced as a consistent failure.
• Trystero 0.22+ changed its API (joinRoom/strategy imports); pin 0.21.8 from a CDN with a fixed version or the app silently never meets peers. Company Chrome on this network returns 0 ICE candidates and the public Nostr relays are blocked, so WebRTC chat/voice cannot be verified here: build a ?local=1 BroadcastChannel test mode for single-browser e2e, show a visible “can’t reach the relays” notice after a timeout, and say plainly that two-device online use is unverified until tested off-network.
• Playwright ‘Connection closed while reading from the driver’ is an environment flake from leaked msedge processes (the smoke test uses channel=msedge with no headless, and timed-out runs leak browsers before close). Clean them first: taskkill the definitive signature –remote-debugging-pipe (the user’s real Edge never has that flag), and you often need a second direct per-PID pass for orphaned Chromium children; then retry. It is not a code bug if the same run was green moments earlier. (correction from the human)
• To verify a clipboard/sharing UI feature in headless Playwright without flaky clipboard permissions, expose a deterministic hook (window.hoki.getShareText()) that returns the composed string and assert on that, plus an is_visible() check on the button; do not assert the Copied label flip (it is clipboard-dependent). PowerShell here uses ; not && as the statement separator. (correction from the human)
• A fortune blessing string that contains an apostrophe (e.g. Don’t / you’ve) MUST use double quotes; a single-quoted string with an apostrophe ends early and throws a SyntaxError that silently breaks the whole app (no paw, blank page, Playwright times out on #hoki-paw). No Node on PATH, so use the Playwright run as the JS syntax verifier. (correction from the human)
• Jeda: headless Edge + SwiftShader stalls when one browser hosts several WebGL pages in a row (world never reports ready); launch a fresh browser per 3D screenshot. Also: pairwise ties are not wins - a +0.40 candidate with 1 win / 2 ties is a stall; park it and re-test with more judges instead of shipping.
• The keep-best rule correctly rejected a round that scored +0.4 on the rubric but tied 0-0-3 in the pairwise check; report the kept score (7.55), not the highest seen. On a loaded CPU, use low render quality + render cap + DOM clicks in tests and 180 s timeouts for high-quality screenshots.
• Nongkrong: Playwright with several pages in one context: call page.bring_to_front() before waiting on a WebGL ready flag; background pages never run requestAnimationFrame. Check that judge screenshots are fresh (file size/time) before trusting a round.

• Never edit UTF-8 source with PowerShell Get-Content

  Set-Content (PS 5 reads as ANSI -> mojibake in CJK/em dashes); edit via Python. cairn vibe end closes the latest live session, not yours, when several agents run - check the file it names.

• Overlay CSS appended to a third-party page loses to its own rules on specificity (e.g. .topbar > div:first-child beat .topbar > .badge); match or exceed the original selector, and screenshot every candidate before the council.
• Pocketdex: a view whose mount() adds listeners to a persistent container stacks them on every render; toggles (Pause) fire twice and cancel. Test a toggle after at least two renders. Fix: replace the container in render().
• Toggle pressed state: a .btn.quiet background rule beat [aria-pressed=true], so the pill turned invisible. Give the pressed style higher specificity and add an on/off dot so state does not rely on colour alone.
• (founder-studio v2) A style gate that clicks a 3D object must check the click reached the canvas (elementFromPoint) and that the selected id matches, or overlays and pre-opened panels give false passes/fails.
• Pin the clock in smoke screenshots (riak: RIAK_SMOKE_HOUR=14). Day/night tint changed between baseline and candidate shots and confounded the council; re-shoot the baseline with the same pin before comparing.
• Reactivecircus/android-emulator-runner runs every script: line in its own sh; a variable set on one line is gone on the next. Put capture-and-use (e.g. uiautomator bounds -> input tap $XY) on one line.
• Deep-scan “clipped” flags can be pre-existing intentional ellipses (shell .meta); fix them anyway (display:none below a breakpoint + margin-left:auto on the next item) so the gate is truly 0 at every width.
• An unclosed @media brace silently scoped every later rule to phones; desktop looked “fine” only because the base styles carried it. Add a brace-balance test to any repo with a hand-written stylesheet.
• A new popover in a sticky header can be hidden under a later sticky sibling; give the header a higher z-index and verify the open state with a functional test, not only the closed screenshot.
• Two Playwright installs (user site-packages vs global) wanted different browser revisions; python -m playwright install webkit (with proxy) fixed WebKit. Check playwright.__file__ first.
• A native iOS build is reachable without a Mac: GitHub macos runner + Capacitor -> simulator screenshot + unsigned IPA for AltStore/Sideloadly. App Store still needs a paid developer account.
• Moving a sticky nav above a stage makes element.scrollIntoView land targets under the sticky bars; set html scroll-padding-top = header + tabs height (also fixes Playwright click timeouts).
• Runtime emoji scan: use the browser regex \p{Extended_Pictographic}; scan shipped files only (skip vendor/test) and run it after, not alongside, the gate run (GPU contention hangs it).
• Android CI with no local SDK: gradle lintRelease is a real gate. The first failure was MissingPermission (VIBRATE). Read the job log (/actions/jobs/<id>/logs) instead of guessing.
• (founder-studio v2) Google Fonts is blocked on this network and Playwright screenshots hang waiting for font loads: self-host fonts (copied from Cairn, OFL) before any audit.
• Council plateau: when r(n+1) ties r(n) pairwise and gains are invisible in screenshots (tooltips, tabular-nums, no-wrap), keep it on objective grounds and stop the loop.
• Headless three.js timeouts can be machine load from other sessions, not a regression: measure frames in 3 s (tests/diag_fps.py) on the previous commit before bisecting.
• Shell tool: a Select-String with no match reports failure; the window.hoki test hook gets overwritten if app.js reassigns it, so extend it rather than replacing it.
• Under machine load (other sessions’ Playwright jobs) single click-timeouts/browser-closed errors appear; rerun that case alone before treating it as a regression.
• When a hone ends with “limitations”, check which ones are cheap to close with data the app already fetches (Nominatim extratags gave opening hours for free).
• Animating (CSS scale) the click target’s SVG child makes Playwright page.click time out (“element is not stable”). Animate a sibling (halo/glow) instead.
• Council crops images taller than about 1440 px: downscale phone screenshots to 1400 px high before judging, or the bottom of the UI is never scored.
• Release APKs rename resources (res/Xy.png). To verify frames, count the large PNGs and check for “APK Sig Block 42”; don’t search for cat_* names.
• First delivery was a 2D floor plan; the user wanted the Jeda/Sansa 3D walkable space. Check the referenced project’s interaction model first. (correction from the human)
• Swipe-velocity tests in px/ms flake when the canvas shrinks or CI is loaded; scroll the target into view and retry like a player would.
• A size gate (44 px tap targets) caught what the judges liked: v9 scored 8.15 but failed G6. Run gates before celebrating a score.
• Fixed sleeps in e2e (wait 2.5 s for a walk animation) flake under load; poll with wait_for_function and keep the same assertion.
• A live region that is re-created on each redraw does not announce repeats. Use one persistent role=status node, created once.
• Media-query overrides lose to earlier selectors with higher specificity; scope phone rules under the panel id (#explore …).
• A top-level const X in a classic script is not on window; feature-check with typeof X !== 'undefined', not window.X.
• Landing copy lives in state/data.json at build time; state/content/.json only applies on ingest (needs gh on PATH).
• Check brace balance of styles.css: one unclosed @media silently scoped every later rule to phones for days. Added a test.
• Full-page 1440 screenshots with live map tiles can time out; reuse the final-check capture instead of re-shooting.
• Windows keeps zombie http.server sockets (SO_REUSEADDR): use a fresh port per gate run instead of reusing one.
• Scope Playwright selectors to .sheet when a sheet is open; the page behind has buttons with the same labels.
• Gate servers need a random port (8791 was taken by another app -> false A1 fail). gates.cmd now uses %RANDOM%.
• Before merging main, fetch: main had independently added the same brace test; keep theirs, drop the duplicate.
• Audit screenshots time out when the CPU is pegged by parallel judges; retry instead of failing the gate.
• cairn vibe check runs a command (vibe check --session ID -- <cmd>); it is not a pass/fail flag.
• Background-attachment: fixed made the audit full-page screenshot time out (30 s). Do not use it.
• Playwright route.fetch() fails behind the local proxy; fulfill from the file on disk instead.
• Toasts can cover buttons in Playwright checks; submit forms with Enter instead of clicking.
• 1) Near 9/10 the council cannot separate versions: v2.1 scored 9.30 then 8.90 on the same text, so a “weakest criterion” of 7 can be one judge’s noise. Average 2 runs before choosing a target. 2) Each fix round makes judges raise new complaints (v2.4 -> v2.5 fell 0.4 while winning pairwise 3-0): stop after one targeted round and turn the objective complaints into gates (G8 one path style, G9 loop implements blocked) - gates decide, the soft score does not move. 3) git with system core.autocrlf=true re-adds CRLF on Windows checkouts; repos holding SKILL.md need .gitattributes * text=auto eol=lf.
• Riak whole-app run: adding the pond shot and pond paragraph to the judged set lifted the baseline 8.25 -> 8.80 (the artifact text moves the score as much as the art does). Scene richness (caustic net, carpet, fishing clouds/reflections) scored 9.25 but the blind pair stayed tie-heavy (A1/B0/T2, then 0/0/3): judges cannot see subtle scene detail at screenshot scale side by side. Next time add 2x crops of the changed regions to the pair packet, or the pair cannot reward art work. Also: when the desktop locks, headless smoke times out; run it detached and retry before counting a gate failure.
• Teduh 1.5.0: to beat a rival on content counts cheaply, expand data from a fixed seed (cultivars, ingredient pairings, colour morphs, biome islands) in one module called at the end of data.js; keep core-progress metrics (sky clarity, q_clean) on the hand-made content only, cap long lists in the UI (60 rows + search), and cull far content by distance. Check a ‘night’ screenshot by the HUD clock: tod 0.93 was 05:30 dawn, not midnight.
• Hoki: ‘widget not as responsive as the web’ = the widget was a flipbook of pre-rendered frames. Fix the cause: host the same live web renderer in a native transparent window (Tauri 2) instead of more frames. No local MSVC -> build on GitHub Actions windows-latest and download the artifact to test; std-only TCP hand-off gives single instance without new crates.
• Software WebGL (swiftshader) in Playwright runs a busy three.js page at ~250 ms/frame and starves input: page.click hangs after “performing click action”. Fix in the app, not the test: measure render cost and, when slow, drop to 1x pixels / no shadows and cap the frame rate so the main thread stays half idle (good for old phones too).
• Bots and council pull in opposite directions on a mature UI: bots reward added explanation (hints, visible options), the council marks the same additions as clutter (r1: 8.0 -> 7.4). Express a fix by replacing or reordering existing text, not by adding elements (r2 kept the council at 8.0 and raised bots +0.33).
• Riak follow-up: adding 2x zoom crops of the changed regions to the pair packet flipped w0 vs w1 from 1/0/2 to 0/2/1. For art rounds, always include crops in the pair. Also: smoke exits 0 even when a flow fails; read its errors list, and cache static canvas backdrops (per-frame scenery broke timed bites).
• /hone on itself, run 2: when the rubric itself was missing a correctness check, adding a gate (rubric v2, G7) settled it in one step where three score rounds could not. Anti-loop wording kept losing ~0.8 with one judge (reads it as permission to stop early); the loop’s own done() break is the guard.
• (riak 1.4.0) Extra canvas texture drawn per element (one stroke per scale arc) slowed headless frames enough to miss timed bites and time out screenshots; batch all arcs into one path and stroke once, and treat a sudden smoke timing failure after an art change as a frame-cost bug first.
• The bots read brief.md on every review. A wrong fact in it (witness “optional”) showed up as a new blocker the moment the UI told the truth (“Required”). Check each brief line against the real app before B init; if it is wrong, fix it, --force a new series and re-review r0.
• /hone on itself: near the ceiling (best 9.55) the +0.3 keep rule rejects real correctness fixes even when the blind pair is unanimous (r3 won 3-0). Encode contradictions as hard gates in the rubric up front, so correctness is decided by the checker, not by noisy scores.
• Mint 3.5/3.5.1: test_core runs js/i18n.js in a sandbox with no location/URLSearchParams; guard browser globals ((root.location && root.location.search) || ""). The local test runner skips it without Node, so CI is the only check - push and read the log.
• A page that hovers at the scroll threshold + a canvas ResizeObserver can ping-pong forever (scrollbar on/off changes width). html { scrollbar-gutter: stable; } breaks the loop; symptom in Playwright is screenshot/click timeouts after a layout change.
• A per-object text cache keyed on (mtime_ns, size) via os.stat is the cheap win for long-lived servers (MCP, local API). Don’t use scandir’s cached DirEntry stats on Windows: NTFS dir entries lag for open files and would break mtime conflict checks.
• (cairn vibe r7) In a view that re-renders on a poll, record
  open state on the summary click (synchronous), not on “toggle” (fires a task later); a re-render in between closes it. Local e2e passed by timing, CI caught it.
• Council evidence is part of the score: element screenshots can crop tall cards and instant scrolls can park content under a sticky header. Capture full-element clips and keep the evidence set identical across rounds.
• (teduh 1.4.0) New world objects must avoid the open ground players already build on: two existing placement tests failed because new heaps sat on the meadow. Also migrate old saves so nothing spawns under a build.
• The absolute council score sat at 8.0 for three rounds while every pairwise vote went 3-0 for the candidate. Use pair votes to decide keep/ship; treat a flat absolute score as a plateau signal, not a regression.
• (mint 3.5.5) Generic words (budget/预算/presupuesto/anggaran) are modifiers in intent scoring: when they are the only evidence for a category and a specific category also matches, the specific one should win.
• Keep only ONE live 3D page open per browser in tests: close each context before opening the next, and use goto(wait_until=”domcontentloaded”) + wait_for_function(isReady) instead of waiting for “load”.
• (mint 3.5.5) Probe a command parser with a matrix (languages x command types x 2-3 natural phrasings), not single examples; the 55-cell matrix found 7 dead phrasings that every unit test passed over.
• Four rounds at 8.6-8.7 against a best of 8.8 = plateau. Park the work on a branch, keep best live, and change the lever (bigger scene/content changes) rather than more phone polish.
• (teduh 1.4.0) A screenshot of every new panel catches render bugs the logic tests miss: an h() one-level flatten had silently broken the Island stats tab for releases.
• (hoki v3.1.1) Council endpoint can time out en masse (WinError 10060) -> ‘no judge answered’; check gateway reachability and rerun instead of treating it as a stall.
• A first-load run on a busy machine can take more than 16 s to deal buyers; a phone_check “idle” mid-shot is start latency, not a bug. Re-run before debugging.
• Keep shell calls < 45 s and python calls < 120 s; run long checks (pytest, playwright, landing publish) via Start-Process with log files and poll.
• A front-door redirect page (app.html -> budget/) drops query params unless it forwards them; test ?lang= through the real entry URL.
• A sanitizer like normalize() silently drops new spec fields; when adding state (spec.lang), whitelist it there and assert it in tests.
• Feature releases are user-directed, not council-gated: gate them on tests + a swapped council pair for “no visual regression”.
• Source-text assertions (test_budget_web greps for a call) break whenever a call signature changes; update them in the same PR.
• Templates lives inside Budget’s Market tab as an iframe: pass state (lang) in the iframe src, not via shared storage timing.
• Playwright page.evaluate("act(...)") hangs when the app function awaits a dialog: fire it with () => { void act(...) }.
• Check that a new top-level CSS rule did not land inside an open @media block (the tip-hide rule silently did nothing).
• Random port 0 can land on a Chrome-blocked port (2049 = ERR_UNSAFE_PORT); bind test servers in 18100-18899.
• When a check-in dialog can be swapped (photo -> note), guard late camera promises with video.isConnected.
• Sample animation checks longer under software GL (26 x 110 ms for one full beckon), never 12 x 90 ms.
• Gradle ‘Could not isolate parameters’ on the Android CI job is a flake; a re-push or re-run fixes it.
• Sync-cards.yml lives in juw12/portfolio, not juw12/projects (apps.json lives in projects).
• Read run_gates JSON with encoding utf-8-sig; values may be bools as well as dicts.
• Insomnia (GlobalProtect watchdog): 1) A “live pass” can be a human: after insomnia’s click, GP’s log showed “user click the tray icon” 10 s later. Verify automation by the target app’s own log (who clicked), not by the end state. 2) GP 6.2 ignores button clicks while its panel is hidden (“Connect button click event when main panel is invisible, ignore this event”); launching PanGPA.exe again shows the panel through GP’s own second-instance path, then WM_COMMAND clicks count. 3) PS 5.1: Get-Content x.json | ConvertFrom-Json emits a JSON array as ONE object, and wrapping it in @() nests it; re-enumerate with @($parsed | ForEach-Object { $_ }). 4) powercfg stores some indexes as REG_BINARY; [int] on byte[] throws inside an if (...) condition, the branch is skipped silently, and a “restore” can report success - verify every write by reading it back and keep the backup unless all items verify.
• Ruang (Tauri 2 Android in CI): Gradle’s rust plugin calls back into the CLI with npm run tauri, so npx @tauri-apps/cli alone fails with ENOENT package.json; add a tiny package.json with the CLI as a devDependency next to tauri.conf.json. Build arm64+armv7+x86_64 so the x86_64 emulator can smoke-test the same universal APK. Generate the PKCS12 key with Python cryptography (legacy SHA1/3DES encryption) when there is no keytool. On Windows, Python write_text() turns LF files into CRLF: check git diff --stat before pushing (a 2-line change showed 2970 lines).
• Ruang r2: score the run’s own baseline in the SAME artifact format before judging a candidate (old 7.4 was a different write-up; same-format r1 = 6.9). Attach the same screenshots to both or note it. Write the test for the physical claim first: ‘lens changes width’ was wrong (width = w_px*H/h_px), depth changes.
• Proxy: the local proxy on port 3128 can stop listening mid-run while its process stays alive; probe the port and fall back to the upstream proxy, passing it to git with -c because ~/.gitconfig pins the local one. The two can swap within an hour; test both with git ls-remote right before a publish.
• When L new dies partway, the page can ship without its gallery card (state/gallery.json has no entry). After publish, grep the live /p/ index for the slug; if missing, run gallery.add_project(...), then L gallery, and push.
• Registering an app means bumping every version file: mobile/package.json and the app’s entry in scripts/apps.json were both missed, and test_v04_account_tour.py checks the latter. Grep for the old version before pushing.
• Speech: clips of 8-18 s never showed that the English engine drops everything after 30 s and that another engine falls apart on long audio; test a speech path with a recording longer than its engine’s limit, in every engine.
• Docs: apply a batch of document edits in memory and save only if every replacement matched once. A document rendered from a template must be changed in the template and re-rendered; the claims check caught a direct edit.
• Tests: do not use 2>&1 on native commands in PowerShell. A log-line filter that matches mid-line eats legitimate output containing the same prefix; in a regex alternation put the longer alternative first.
• Live check: comparing served HTML with a local file needs line endings normalised first. A banned-phrase scan for “#1” matches CSS colours; print the surrounding text before treating a match as a claim.
• Test culling with real camera files plus synthetic blurred copies; synthetic demos hid two bugs (median blur bar collapses when most of a batch is soft; 9x8 canvas hash is point-sampled and unstable).
• Before proposing or building, read the latest replies and their attachments (here Lotus’ action-plan xlsx). Mirror the official plan, and mark your own additions as “proposal – not in plan yet”.
• Words: a safety corpus of real transcripts can pass while one hand-written adversarial phrase exposes an older rule; probe fuzzy rules with near misses and a dictionary sweep before trusting a zero.
• Run: when a round passes the gates and wins the pairwise but is under MIN_GAIN, keep main on the best version, land only the run log, and park the candidate as a draft pull request for the owner.
• Research: some servers send gzip bodies unasked; check for the gzip magic bytes before searching the text. ACM and IEEE refuse scripts, but the OpenAlex API returns the abstract for the same DOI.
• Landing: <footer class="wrap"> lost its vertical padding because .wrap { padding: 0 24px } beats footer { padding: ... }; compare selector specificity before blaming the screenshot.
• Emulator: a test that executes odd privileged instructions can crash a browser emulator (v86 fails on a ring-3 iret to ring 0); run the demo’s exact command set in that emulator early.
• Run: check which file a path variable names before printing line ranges from it, and what a helper returns before passing it on: a write() that returned the texts was read as paths.
• Windows Defender quarantines files that contain real attack strings (test fixtures for a security scanner). Store malicious samples base64-encoded and decode them in the test runner.
• A normal-priority background job on this laptop is starved about 10x. Set above-normal priority and turn off power throttling before a timed run, and run a 1-minute speed test first.
• QEMU: on a loaded Windows host its threads get starved and a wrapping PIT clock falls behind; start QEMU at above-normal priority and check guest time against host time.
• Live check: after publishing, boot the live URL in a real browser and compare the served image hash with the local one; a 200 status alone does not show the demo works.
• Landing state/data.json reuses a project’s subtitle inside other projects’ entries; scope text replacements to the slug’s own span and verify only that slug changed.
• Card: on the live portfolio page the new card was hidden at load and appeared after its category filter was clicked; a visibility check must click the filter first.
• Words: a count of “term appears more often than said” cannot see a dropped possessive; also assert that sentences which already contain the term come out unchanged.
• Playwright here needs PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS=1 and channel="msedge"; under a strict CSP pass arrow-function predicates, not strings.
• Emulator: a demo that skips a test in one emulator reports a different count there (46 in the browser, 47 in QEMU); quote each number with its environment.
• Tests: a load-sensitive test that failed under heavy load passed on a re-run; re-run before calling it an environment condition, and record both results.
• Seed pages get a “live” status pill even with no live app; set "status": null in state/content/<slug>.json and re-ingest for source-only projects.
• Proxy: a publish that outlives one tool call keeps running; read its log in the next call and restore temporary settings only after it has finished.
• A unit test on the architect fixes caught a real placement bug (turned desk landing outside the room): test every one-tap fix’s resulting geometry.
• Tests: a helper that maps “could not ask” onto an ordinary error code can make a check pass for the wrong reason; return a value no check expects.
• Tests: a wiring test for code that starts threads and reads the clipboard needs a synchronous thread stand-in and a patched clipboard function.
• Detached council runs can die silently in seconds; give them their own stderr log and check that the out file appears before polling for long.
• PowerShell Select-Object -Last N buffers the whole stream, so you see no live progress. Write to a log file and tail it instead.
• Browser timings taken under load are useless, and a browser check right after heavy tests can flake. Re-run it alone when idle.
• capture.py used Playwright’s bundled Chromium, which a Playwright update removed; it now uses system Chrome like shoot.py.
• Research: a citation checker that reports “confirmed” for an empty phrase list is vacuous; require a phrase for every source.
• Scope ‘no foreign request’ checks to the page under test; a handoff link into another app brings that app’s network calls.
• Tests: designing a test group down to the source before editing let 12 new checks compile and pass on the first build.
• Tests: the test tool’s saved serial log holds the full output of a failing check; read it before re-running.
• Tests: when local tests depend on an optional package that CI lacks, run them once with the package hidden.
• Windows: a helper that takes env= must merge it into the parent environment; replacing it drops PATH.
• A Playwright upgrade can drop its bundled headless shell; tests now fall back to the installed Chrome.
• Check the built exe itself: tauri.conf window title is a separate copy of the tagline and goes stale.
• Python tool dies at 120 s: run gates, card capture and landing publish detached and poll their logs.
• Two Playwright runs at once flaked (shared Edge profile / ports). Run browser checks one at a time.
• Tests: prove that new tests discriminate: run them against the old code and confirm that they fail.
• Bench: smoke-test a long benchmark end to end on a tiny made-up input before the 15-minute run.
• Smoke misses timed bites while the desktop is locked; retry before calling it a failure.
• Render on demand (no idle RAF loop) keeps headless/SwiftShader gates fast and stable.
• Tests: a test that searches a script for a forbidden command must skip comment lines.
• Never start a second gate run while one is alive (same port); check the pid first.
• Council: single-run absolute scores can drift down across rounds while pairwise prefers the newer version 3-0 each time (8.95 -> 8.75 -> 8.60, v2>v1, v3>v2). Decide keep/revert on pairwise, report both, and stop when the fixes list moves to the next tier instead of chasing the absolute number.
• (cpu-eco-toggle) The landing builder writes “Try it live” CTAs and a “try it in the browser with nothing to install” FAQ whenever links.live exists. If the live link is only an illustration, override cta, shot_kind and faq_drop in state/content/.json.
• Council: a scoring run that lands after the decision can move the median (v1 was 9.1 on three runs and 8.875 once the fourth run, 8.15, landed). Wait for every launched run before logging a score or deciding, and re-check the decision when one lands late.
• Landing: a hand-set tests number needs its counting rule written next to it (Glow: 25 test files named in CI plus suite.js run through test_glow_harness.py gives 26; the profile’s 28 counts every file in the two test folders).
• (mlbb-blame r6-7) With the screen locked and other jobs at 100% CPU, a shell can take minutes to start and a push took 9 minutes; check Win32_Process command lines before assuming a hang, and do not kill or retry.
• (kilat r1) A scratch clone’s main goes stale fast; for a one-sentence edit of a shared file (apps.json, profile README) use the contents API against the live file with an exact-count replace, not a local commit.
• (inti r5) A held-out sample stops being held out once tests are aimed at it. Draw a fresh one and keep the old result as “before” evidence: the fresh sample caught 73%, the two aimed-at samples 88% and 95%.
• (inti r5) A surviving mutant is first a question, not a bug: six of round 4’s seven “holes” were missing tests and one was a mutant that changes nothing observable. Read the code before promising a fix.
• (inti r5) A judge that times out: bound its effect instead of rerunning. The bound may not settle the decision (in round 5 it sat above the bar), so say so and decide by the rule written beforehand.
• (inti r3) shutil.copytree keeps time stamps, so cargo may skip the rebuild of a mutated file. Copy with shutil.copyfile and check that the built program is newer than the changed source.
• (kilat r1) The landing shoot step captures only the viewport; use a tall headless-browser window to see the lower sections (it showed a clipped quick-start line the gates did not catch).
• (inti r3) A mutation sample that tests were written against is no measure (58 of 60). Draw a held-out sample and report it as it comes out (45 of 60, with 9 real holes).
• (inti r4) Before building a “missing feature”, test whether it already works; selective revocation needed a document correction and a test, not kernel code.
• (inti r4) One short baseline under an emulator on a busy host is not a reference; take several and say which one is used. Report the changed check.
• (mlbb-blame r6-7, 2026-11-12) offsetParent is non-null inside a closed <details>; use checkVisibility() to test what the user can see.
• (mlbb-blame r6-7) Gate scripts run on file:// with no proxy, so background network warm-ups must be gated on a service-worker controller.
• (inti r4) Read the full serial log before diagnosing a failed gate; the one-line gate summary is cut short and pointed at the wrong cause.
• (mlbb-blame r6-7) A pairwise win can come with a lower single absolute score (v8 9.0 vs v7 9.3); log which rule decided the keep.
• (kilat r1) Cairn notes open in Write mode, so a browser test must press Preview before looking for rendered plugin blocks.
• (mlbb-blame r6-7) The landing check Page.goto 30 s timeout was machine load (100% CPU); the re-run passed unchanged.
• (mlbb-blame r6-7) Playwright inner_text is empty inside a closed <details>; read textContent or open it first.
• (mlbb-blame r6-7) Run Playwright scripts from the user’s shell so the same install and browsers are used.
• (inti r6) A mutant “caught” in QEMU through one timing check is not a catch until it is repeated beside the unchanged kernel: here the mutant failed 2 of 7 runs and the unchanged kernel 1 of 7, in the same subtest, so the catch was not counted and a flaky check was found instead.
• landing.py crashes printing ingest output on cp1252 consoles: set PYTHONIOENCODING=utf-8. state/content/<slug>.json steps must be {title, body} objects, not strings. A project with no live URL needs shot.jpg placed by hand (a README screenshot works).
• Check the repo’s first CI run before writing “CI passes” anywhere: a local pass on Python 3.13 hid a 3.10 failure (pip resolved cadquery-ocp 7.x, which lacks OCP.collections).
• (inti r6) A watcher’s --until pattern must not match ordinary test lines, or the wait ends at the first test and the job is read as finished while it still runs.
• An edit() find-and-replace helper needs anchors that occur exactly once; assert the count before replacing.

This site is open source. Improve this page.