leanwheel-skills, Round Four: Fewer Guardrails, More Proof

This is the fourth post about leanwheel-skills. The first covered why I built it, the second covered the agentic-engineering layer, and the third covered the doc-free single-goal lane. Three weeks later, the theme is trust: trusting the model with fewer explicitly defined rules, trusting the gates only after they've proven they can fail, and trusting the knowledge base only when I know where each claim came from.

Aligning with Anthropic's skill guidance

I realized a few weeks back that Anthropic has been publishing best-practice guidance on how to author skills, and the newest model generation changed some of that guidance: over-constrained prompts now cost you capability, not just tokens. So... I audited every skill in the repo against that guidance, and PR #21 is the result. The audit found a pattern I'd been building up since I started: I was writing conduct rules in contract voice.

The distinction is now a first-class authoring rule in the repo (recorded as DD-02):

Every MUST, never, and HALT in the skill files got re-justified against that table. Contracts stayed hard, because repeatability across sessions depends on those shapes staying stable. Everything else got rewritten as plain guidance that trusts the model's judgment. The measurable outcome: emphatic-language dropped noticeably in the densest skills, with a contract grep confirming all 58 contracts survived - zero dropped.

The second change from the same audit: each rule now has one canonical home. The fix-now ceiling, the fail-first rule, the testing-plan format - all of them used to be restated in multiple skills. And what happens when the same text shows up in multiple places? Drift. Now, one skill owns the text and the others cite it ("per dev-story → Testing Plan"). Same idea as single source of truth in documentation, which regular readers know I'm slightly obsessed with.

Third: the rationale moved out of the skills entirely, into a design-decisions log with numbered entries (DD-01 through DD-67 so far). Skills cite DD-NN instead of restating why; you read the entry only when you need the why. Two nice side effects: the sync-from-upstream notes collapsed from a 387-line history dump to a preserve-on-sync table, and the log gave me a place to enforce a rule I'd been sloppy about - shipped files never name the project a lesson came from. Lessons are recorded generically ("a brownfield SwiftUI + Core Data project"), which matters for a framework designed so other people can install and use it.

The same single-source logic then got applied to Anthropic's own docs. The repo's CLAUDE.md gained a reference-pointers section - skills authoring, prompt caching, context editing, compaction, tool use, structured outputs - with a hard rule: fetch live, never summarize. A cached summary of a fast-moving best-practices page is a stale opinion wearing a citation. The pointer costs a handful of tokens; the summary would cost correctness.

Finally, the repo now has evals for its own skills, using the claude plugin eval layout. If the framework preaches "a required output is a named report field plus a zero-token check, not more prose about how to produce it," it should hold itself to that. Changing a report field or a parsed heading now means updating the matching eval case, or the change doesn't land.

That re-verify idea got applied retroactively to the biggest empirical claim in the repo: model routing. "Sonnet loops on Swift builds, so route Swift dev to the big model" was true in the Claude 4 generation, but it's a claim about a model generation, not a law. DD-20 now dates the rationale and points at the evidence that would change it - the flywheel ledger records build-and-test iterations per model, and the retrospective reports them per epic. Change the routing when the numbers say so, not the prose.

Gates that can prove they can fail

The other big thread (PR #19 and #20) came out of a painful retrospective finding on a real project: one 14-story epic shipped five gates that could not fail. A grep over an empty file walk. A test asserting on the summary line of a different target. Each one green, every run, forever - and green-that-can't-be-red is worse than no gate, because it builds confidence where there should be none.

The fix is a fail-first rule: a new test, eval, or assertion counts only once it has been shown to fail. Break the thing it guards, watch it produce a named failure, restore, watch it go green. And because "please remember to do this" is exactly the kind of conduct prose the other half of this post argues against, it's mechanical: a new sabotage.sh script does the break/verify/restore cycle deterministically, and dev-story and code-review invoke it rather than promising to be careful. Related rule in the same spirit: enumerating gates ("all 12 screens have identifiers") must assert a lower bound, because a gate that finds zero of something can't tell coverage from a broken walk.

Two follow-ups landed the week after. A verify-green rule in code-review: a blocked verification is a red, full stop - there is no such thing as a "qualified PASS." And the flywheel's metrics ledger became a script (ledger.sh) instead of a prose instruction to append a JSON line, for the same reason the epic context cache became a checked file back in round three: bookkeeping the model narrates is bookkeeping that doesn't always happen.

The finding-routing half of #19 fixed a quieter failure. Review findings used to have two dispositions, patch or defer, and trivially-fixable items that fell outside the story's ACs defaulted to the deferred log - which grew from 13 to 27 open items in a single epic. There's now a third disposition, [Fix-Now], with a strict four-condition ceiling (roughly: ten lines, one file, adjacent to the diff, provably safe, no schema or API change). Small enough to fix on the spot, recorded so it stays reviewable, and the deferred-items intake now rejects anything that meets the bar. The ledger is for real decisions, not for procrastination with a paper trail. I honestly felt like I was simply creating a laundry list of "deferred" items every story, and it seriously needed to stop.

Field knowledge is not research knowledge

The newest work (PR #32) fixed a failure mode I hadn't seen anyone name. The framework's Swift and web guidance folders carry two kinds of claim with opposite aging behavior, and until now nothing distinguished them.

Research knowledge comes from Apple docs, WWDC sessions, release notes. It goes stale on the OS cadence, and a newer primary source supersedes it outright. Refreshing it is exactly what the /refresh-swift and /refresh-web skills exist to do.

Field knowledge comes from trial and error on a real shipping project - a mechanism observed, a symptom paid for, a fix verified by running. It's typically absent from any primary source, which is precisely what makes it valuable, and it does not go stale on a version bump. It retires only when a source shows the underlying mechanism changed. Think of it like building your own "stack overflow" of lessons-learned.

Left undistinguished, an automated research pass eventually flattens the second kind: it finds current guidance on the same topic, rewrites the section, and the rule that cost three debugging sessions to discover disappears with no diff anyone reads as a loss. The failure is quiet and one-directional. And it also turns into wasted turns to "rediscover" it later.

The fix (DD-66) is a cheap mechanical marker: field-earned rules carry an HTML comment (<!-- FIELD: ... -->) - invisible when rendered, greppable when not. A refresh pass may append a dated verification note to a FIELD block, version-scope it against a citation, or retire it with the citation showing the mechanism is gone. It may never silently rewrite one because newer general guidance covers the same topic. The bar is the mechanism, not the topic: "Apple now recommends X" does not retire a rule about what happens when X breaks. When a refresh finds a conflict it can't resolve to that bar, it reports and changes nothing - the person who paid for the rule decides.

The companion change (DD-67) fixed the delivery side. The guardrails block that gets scaffolded into a project's CLAUDE.md used to be a plain copy-paste that diverged forever - and since CLAUDE.md loads every turn, the stale copy is the one that wins. The block is now a versioned managed region (<!-- leanwheel:guardrails swift vN -->) holding a pointer to the reference docs plus only the tripwires that must be known before a plan is formed. /upgrade-project reads the version: older gets refreshed in place, unversioned (hand-edited, pre-managed) gets diffed and offered, never overwritten.

The App Store pipeline kept finding silent failures

Round three's simulator automation is now driving real App Store submissions, and adoption on a real project surfaced a cluster of failures that share one trait: nothing anywhere reports them.

The scariest is CloudKit schema deployment (DD-64). NSPersistentCloudKitContainer ships no schema file - it infers record types from your model and creates them lazily in Development, on first save of that type. Put another way, the Development schema is whatever manual testing happened to touch. The "Deploy Schema Changes" only copies what Development already has, and an entity never exercised on a dev-signed device is simply absent in Production. No joke - it FAILS for the first real user who creates one and never for the developer. The stubs now carry the initializeCloudKitSchema pattern for both Core Data and SwiftData, and /appstore-preflight raises a HIGH finding on any CloudKit container without it.

Three more in the same family, each now either impossible or a hard stop:

• Debug UI in store screenshots. Capture runs require a Debug build (seeding and routing are #if DEBUG), so every debug-only view is a standing screenshot contaminant. I ran into then when testing found one leaked view rendered only during seeded capture runs - invisible in ordinary testing and present in exactly the images bound for the App Store. Whoopsies! Store captures now run with an implicit release-parity flag that gates every debug affordance off, enforced by a source-walking test.
• Orphaned screenshots. The composer writes to deterministic paths, so renaming or reordering a row in the plan left the old file behind. This meant you had a tracked, unmodified file, invisible in git status and in every PR diff. The composer now prunes its own stale output (only files matching its own naming pattern, only after validation succeeds, and it logs the count - a tool deleting files silently inside your repo is its own failure mode).
• Stale vendored scripts and ambiguous simulators. When I loaded my project into the Xcode Developer Beta (for iOS27 compatibilty checking), I discovered the sim script is copied into projects and never auto-updated, so a stale copy is the steady state; skills now grep the project's copy for the exact flags they invoke and hard-stop with "run /upgrade-project" instead of failing weirdly. And a device name matching two installed OS runtimes is now a named hard stop rather than a silent wrong-OS run (as is what happened with I had two "iPhone 17" sims - one running 26.6.1 and another running iOS 27).

Smaller things worth a paragraph

Test plans stopped double-billing. The per-story testing plan now splits into AUTOMATED and MANUAL sections, and the epic-boundary manual test plan subtracts the steps already covered by automation, listing them under an explicit "Automated - do not re-test" header. I was the manual tester re-verifying things a script verifies every run. There's also a new plan-defect finding kind, because sometimes the bug is in the test plan, and the retro now counts those too.

Planning consolidated into a decision loop. PR #18 collapsed five planning skills (brief, forge-idea, prd, ux, architecture) into a recursive loop: ideate pulls decisions out of you one at a time, decision-log records them, and spec renders the documents from the log. The docs became what they always should have been - projections of the decisions, not the place decisions hide.

Quiet toolchain output. Tool output persists in the conversation and gets re-sent every turn, and on Swift a single verbose xcodebuild run can outweigh the whole story it was verifying. Build and test gates now run with quiet flags, tee the full log to a self-ignoring .leanwheel/logs/ directory, and keep only the tail in context. Same shape as every other token fix in this framework: the information isn't gone, it's just not riding along in every request.

Docs for humans, not just agents. The guide grew a symlinking explainer (the copy-vs-pointer distinction between plugin installs and live symlinks, and who should pick which) and an ELI5 walkthrough of spec-driven development with an artifact-flow diagram - because "how do the pieces connect" was the question every new user asked first.

An App Store Connect lane. New /appstore-connect skill authors the store inputs (framed screenshots at exact store sizes, listing copy, an IAP spec reconciled against the actual StoreKit config) as lint-clean committed artifacts, building on the simulator automation from round three. Niche, but it turned a dreaded afternoon of clicking into a reviewable diff.

The repo is at github.com/rterakedis/leanwheel-skills. Still MIT licensed. Comments welcome!