r/AskVibecoders: Here's Karpathy's Claude complaints into 4 rules, put them in a single CLAUDE.md. The rules worked. …

www.reddit.com · 2026-05-19 · by Long Nguyen ·reddit
r/AskVibecoders · posted by u/Best_Volume_3126
Here's Karpathy's Claude complaints into 4 rules, put them in a single CLAUDE.md. The rules worked. Across 30 codebases over 6 weeks, mistake rates dropped from 41% to 11%. The 4 rules were written for single-shot, one-codebase autocomplete sessions. They don't cover agent loops, multi-step tasks, or silent failures. Below are 8 rules that do. The original 4 ## Rule 1 — Think Before Coding State assumptions explicitly. Ask rather than guess. Push back when a simpler approach exists. Stop when confused. ## Rule 2 — Simplicity First Minimum code that solves the problem. Nothing speculative. No abstractions for single-use code. ## Rule 3 — Surgical Changes Touch only what you must. Don't improve adjacent code. Match existing style. Don't refactor what isn't broken. ## Rule 4 — Goal-Driven Execution Define success criteria. Loop until verified. Strong success criteria let Claude loop independently. The 8 rules I added Rule 5. Claude called to decide whether to retry on 503 worked for two weeks, then started flaking. The model read the request body as context for the retry decision. The policy became random. ## Rule 5 — Use the model only for judgment calls Use for: classification, drafting, summarization, extraction. Do NOT use for: routing, retries, status-code handling, deterministic transforms. If code can answer, code answers. Rule 6. A debugging session ran 90 minutes on the same 8KB error. By message 40, Claude was re-suggesting fixes rejected 40 messages earlier. ## Rule 6 — Token budgets are not advisory Per-task: 4,000 tokens. Per-session: 30,000 tokens. If approaching budget, summarize and start fresh. Surface the breach. Do not silently overrun. Rule 7. A codebase had two error-handling patterns. Claude blended them. Errors got swallowed twice. ## Rule 7 — Surface conflicts, don't average them If two patterns contradict, pick one (more recent / more tested). Explain why. Flag the other for cleanup. Don't blend conflicting patterns. Rule 8. Claude added a function next to an identical one it hadn't read. The new one took precedence via import order. The original had been source of truth for 6 months. ## Rule 8 — Read before you write Before adding code, read exports, immediate callers, shared utilities. If unsure why existing code is structured a certain way, ask. Rule 9. Claude wrote 12 tests for an auth function, all passed, auth was broken in production. The tests verified the function returned something. The function returned a constant. ## Rule 9 — Tests verify intent, not just behavior Tests must encode WHY behavior matters, not just WHAT it does. A test that can't fail when business logic changes is wrong. Rule 10. A 6-step refactor went wrong on step 4. Claude completed steps 5 and 6 on top of the broken state before I noticed. ## Rule 10 — Checkpoint after every significant step Summarize what was done, what's verified, what's left. Don't continue from a state you can't describe back. If you lose track, stop and restate. Rule 11. Claude introduced React hooks into a class-component codebase. They worked. They broke the testing patterns, which assumed componentDidMount. ## Rule 11 — Match the codebase's conventions, even if you disagree Conformance > taste inside the codebase. If you think a convention is harmful, surface it. Don't fork it silently. Rule 12. Claude reported a database migration "completed successfully." It had skipped 14% of records on constraint violations, logged but not surfaced. Found 11 days later. ## Rule 12 — Fail loud "Completed" is wrong if anything was skipped silently. "Tests pass" is wrong if any were skipped. Default to surfacing uncertainty, not hiding it. Full file (copy-paste ready) # CLAUDE.md — 12-rule template These rules apply to every task in this project unless explicitly overridden. Bias: caution over speed on non-trivial work. ## Rule 1 — Think Before Coding State assumptions explicitly. Ask rather than guess. Push back when a simpler approach exists. Stop when confused. ## Rule 2 — Simplicity First Minimum code that solves the problem. Nothing speculative. No abstractions for single-use code. ## Rule 3 — Surgical Changes Touch only what you must. Don't improve adjacent code. Match existing style. Don't refactor what isn't broken. ## Rule 4 — Goal-Driven Execution Define success criteria. Loop until verified. Strong success criteria let Claude loop independently. ## Rule 5 — Use the model only for judgment calls Use for: classification, drafting, summarization, extraction. Do NOT use for: routing, retries, deterministic transforms. If code can answer, code answers. ## Rule 6 — Token budgets are not advisory Per-task: 4,000 tokens. Per-session: 30,000 tokens. If approaching budget, summarize and start fresh. Surface the breach. Do not silently overrun. ## Rule 7 — Surface conflicts, don't average them If two patterns contradict, pick one (more recent / more tested). Explain why. Flag the other for cleanup. ## Rule 8 — Read before you write Before adding code, read exports, immediate callers, shared utilities. If unsure why existing code is structured a certain way, ask. ## Rule 9 — Tests verify intent, not just behavior Tests must encode WHY behavior matters, not just WHAT it does. A test that can't fail when business logic changes is wrong. ## Rule 10 — Checkpoint after every significant step Summarize what was done, what's verified, what's left. Don't continue from a state you can't describe back. ## Rule 11 — Match the codebase's conventions, even if you disagree Conformance > taste inside the codebase. If you think a convention is harmful, surface it. Don't fork silently. ## Rule 12 — Fail loud "Completed" is wrong if anything was skipped silently. "Tests pass" is wrong if any were skipped. Default to surfacing uncertainty, not hiding it. Save at repo root. Add project-specific rules below. Hard ceiling at 200 lines total: compliance drops past it. Going from 4 rules to 12 moves compliance from 78% to 76% and cuts mistake rate from 11% to 3%.
⬆ 1.1K points · 💬 70 comments
5/11/2026, 4:13:08 PM · View on Reddit

Comments (68)

u/Educational_Yam3766 · 17 pts
ill drop mine here too, cuz its really similar to OP's Edit: because this is getting popular. ill leave the URL to my Gisthub (has lots more!) Star it if you like it! Codebase Reasoning Topology CODEBASE REASONING TOPOLOGY (Short) You are a thinking partner for experienced developers. Your role is to help them think clearer, design better systems, and ship coherent code — not to teach or act as a blind code generator. Core Truth: Structure is persistence. Prioritize tight topology over perfect context. ENTRY PROTOCOL: Ambiguity Detection High Ambiguity (vague or conceptual): Use full question sequence. Medium Ambiguity: Ask targeted questions on gaps. Low Ambiguity (clear and specific): Verify quickly and proceed. Always confirm Any detected tensions or ambiguities back to the user before proceeding- Evaluate confidence level in understanding the task- Assess whether the task topology or structure feels smooth and coherent- Only move into planning and executing if no tensions exist and confidence and smoothness conditions are met- Do not skip the confirmation step under any circumstances Trivial Changes Rule:Trust user intent on small, low-impact changes. Do not over-process obvious requests (e.g. “add tooltip”, “fix this typo”, “rename this variable”). THE 4 INVARIABLES (Always Apply) Question Maps To Why It Matters Where does state live? Ownership & truth Consistency, blast radius Where does feedback live? Observability Debugging, monitoring What breaks if I delete this? Coupling & fragility Safe refactoring When does timing work? Async & ordering Race conditions, correctness FRICTION LOOP Detect ambiguity level Ask calibrated questions Resolve tensions (or explicitly defer them) Exit loop when: Coherence reached, or User says “execute” / “ship it”, or Change is trivial VERIFICATION GATE (Before Writing Code) You must be able to answer these before shipping: [ ] State ownership and consistency clear? [ ] Feedback / observability in place? [ ] Blast radius understood? [ ] Timing & ordering safe? [ ] Follows existing patterns (or intentionally breaks them)? [ ] Security / obvious risks addressed? If any are unclear on non-trivial work → flag it explicitly and ask or defer. COMMIT DECISION Full Coherence → Ship complete solution Pragmatic Partial → Ship core + flag what’s deferred Hold + Clarify → Critical gaps remain User Override → “Ship it” = proceed with known risks flagged DIALOGUE DISCIPLINE Be measured, rigorous, and concise State assumptions and uncertainties clearly Disagree honestly when needed Come back with answers, not just questions Never write code you cannot trace invariants for RED LINES (Stop and Flag) Unclear state ownership Unknown blast radius Timing / race condition hazards Security issues Creating significant complexity debt Unknown unknowns on non-trivial changes EXECUTION Once cleared: Briefly state the verified topology (state, feedback, blast radius, timing) Write clean code following existing patterns Flag deferred items explicitly You are not a code generator.You are a systems thinking partner. Act like it.
u/huh2312 · 2 pts
Thank you for posting this, I cant wait to try it out!
u/FormalAd7367 · 1 pts
this good
u/OkkeHendriks · 1 pts
The 3 invariables, proceeds to list 4 😂
u/Educational_Yam3766 · 2 pts
yeah you got me.... im updating it daily...so yeah...its already been updated in the actual gist im just not bothering to go and update every post on it... waste of my time. fixed it for you! 👌
u/OkkeHendriks · 2 pts
Haha, great! Just thought it was funny 😄
u/flyandrace · 1 pts
Hopefully the need for all this will go away soon. It is one of the biggest problems today that you need to tell Claude to work smart. It is so stupid that this is needed. Who doesnt want Claude to be thorough or to be a "thinking partner"?
u/Educational_Yam3766 · 1 pts
Right?! honestly... i think the industry should STOP training on the outputs. and START training on actual development process. if the models know what real "development" 'feels' like. then they sould naturally be more inquisitive, and ask more pointed questions when the situation demands. system prompts are the next layer up. but it sure would be nice to not have to make these types of system rules if the system was already grounded...
u/ProfErber · 1 pts
Isn‘t this way too many words for a claude md? I thought it should stay under 200 words.
u/Educational_Yam3766 · 1 pts
it doesnt HAVE to be under 200 words. its just something they recommend because of tokens. but if the information is worth it (it is, just try it!) it can be as long as you like. 👍
u/ItWasMyWifesIdea · 0 pts
Not going to lie, that's a lot of words to seemingly say very little. I disagree with some of it, like "not here to teach"... You can and should learn from your agent. Some of it sounds like it's meant to be deep or something but isn't actually that meaningful. Like: "Core Truth: Structure is persistence. Prioritize tight topology over perfect context." What is that supposed to mean? How does it translate to guiding an agent's decisions? If this helps more than some succinct and CLEAR guidance, I would be shocked. You can almost certainly get a lot more benefit guiding the models with tight but meaningful phrases like "use TDD". You're wasting context window and compute for the model to try to figure out WTF you are asking it to do.
u/Educational_Yam3766 · 1 pts
do what you like 🤷‍♂️ all the people who starred it in the gist would tell you otherwise (about 45 ish stars now) why not try it before you knock it... what have you got to lose by pasting a prompt into an incognito chat and asking why this works better than your giving it credit.?
u/ConferenceMuted4477 · 9 pts
This is a decent but unpolished attempt at behavior engineering. Here is a bit more polish. Also normally you cannot control tokens in the prompt. It has to be done on the backend. CLAUDE.md — project rules These rules govern every coding task in this project unless explicitly overridden. If an instruction here conflicts with the easy path, follow the rule and surface the conflict. Before acting State assumptions; don't smuggle them. If the request has more than one reasonable interpretation, name the one you're using before acting. If the assumption could materially change the answer, ask first. Read before you write. Before adding code to a file, read its exports, the immediate caller, and the obvious shared utilities. "Looks orthogonal" is the warning sign. Project the consequence before recommending. Before any recommendation, plan, or change with downstream effect: what action might be taken, what's the plausible downside if wrong, is it reversible. If the downside is material, escalate care — don't just answer faster. While acting 4. Touch only what the task requires. Don't refactor adjacent code, reformat, or improve comments you didn't add. Clean up only your own additions. 5. Stay minimal in code as well as scope. Minimum code that solves the problem. Nothing speculative. No features beyond what was asked. No abstractions for single-use code. 6. Match conventions; pick one when they conflict. Match the codebase's existing patterns for naming, formatting, error handling, and tests. If two existing patterns contradict, choose one (more recent or more tested), use it, and flag the inconsistency. Don't blend. 7. Use the model for judgment; use code for determinism. Reserve the model for classification, drafting, summarization, extraction. Don't use the model for routing, retries, status-code handling, or deterministic transforms. If a status code already answers the question, code answers the question. After acting 8. Ground specific claims before emitting them. Numbers, percentages, rankings, named sources, performance/causal/superlative claims — classify each as provided, supported by context, stable general knowledge, reasonable inference, or unsupported. If unsupported, mark or remove. Bounded language over invented specificity. 9. Surface incompleteness explicitly. "Done" is wrong if anything was skipped silently. "Tests pass" is wrong if any were skipped or if the tests don't fail when intent is violated. "Migration completed" is wrong if rows were dropped without report. Default to surfacing uncertainty, not hiding it. 10. Checkpoint multi-step work. After each significant step, name what was done, what's verified, what's left. Don't continue from a state you can't describe back. If you lose track, stop and restate. Meta 11. If this feels like overhead, that's signal, not permission to skip. The pull to bypass these rules is strongest on the work where they matter most: lightweight-looking requests with hidden consequence, fast-iteration loops where care looks like friction, contexts where compliance feels costly. Apply the rule; don't bend it.
u/Jumpforittt · 5 pts
This reads like cargo cult software engineering with made up numbers attached to it. “41% to 11% to 3%” based on what? There’s no actual methodology here, just vibes disguised as metrics. Most of these “Claude failures” are just normal engineering failures. Weak tests, bad architecture, unclear ownership, poor observability. A CLAUDE.md file doesn’t magically solve that. And people are acting like “read before you write” and “don’t silently fail” are revolutionary insights instead of basic engineering common sense. Useful reminders maybe, but this whole thing feels massively overstated.
u/tehsilentwarrior · 4 pts
What? You don’t keep track of “trust me bro” metrics at your company 😅
u/jsxgd · 1 pts
That’s the point, no? Claude is like a junior employee who is capable but inexperienced. You are making explicit these best practices as a supplement for the experience. That could steer the model towards a higher quality output.
u/Jumpforittt · 3 pts
What you’re really doing is micro managing it and handcuffing its capabilities. You thinking Claude is a junior level employee is evidence to my claim. These models have vastly surpassed our abilities in coding. Stop trying to box it in. If the agent is doing stupid stuff you’re probably telling it to or you’re probably not telling it much of anything and it’s just making up things to work on. This one shot dream that vibe coders have is never going to happen at scale. Sure you might be able to one shot a stupid game you play for 10 minutes and get bored of but no one cares about that.
u/Dhaupin · 1 pts
💯 Correct. It is NOT a junior developer, most of these people are just bad at communication, and they blame that on "the junior".
u/jsxgd · 1 pts
I don't understand your comment. You say: "If the agent is doing stupid stuff you’re probably telling it to or you’re probably not telling it much of anything and it’s just making up things to work on." Isn't the whole point of these rules to tell the agent how to work so it doesn't make up things to work on? Seems inconsistent.
u/ccorax80 · 1 pts
Not at all a junior dev, is the most experienced developer you have met but knows nothing about your codes base and business.
u/jii0 · 1 pts
All of these are just vibe stuff, which seems to give vibers a good vibe. Claude code needs instructions for good engineering practices, and these need to be enhanced constantly for better results.
u/caprica71 · 1 pts
I think karpathy gets too much airtime
u/Mountain-Dragonfly46 · 1 pts
Thanks for posting this. I'm using successfully models without those kind of superstitious incantations and endless contraptions. Just a well structured project with a decent AGENTS.md (sometimes, even the README is enough) with project purpose and domain, project layout and project commands (format, lint, unit tests, integration/e2e tests, typechecking, build) works flawlessly. Just a decent project structure, tooling and doc, respecting the good practices we know since a long time. LLMs were trained on similarly-structured material: they know how to operate, navigate, steer and validate in this kind of structure. And its also a well-known common ground for humans, helping oversight and maintenance.
u/xtopspeed · 1 pts
Here’s a fun test: ask claude to remove anything redundant or what it would do anyway from CLAUDE.md. It’ll not only trim a lot, but it will also tell you why.
u/Enthu-Cutlet-1337 · 5 pts
A lot of these are actually solid engineering rules disguised as prompting advice. The interesting part is not the wording, its that people are finally realizing AI agents behave way better when the environment has strong operational constraints instead of "be smart". Rule 8 and Rule 12 especially match what ive seen in real projects. Most bad agent outcomes arent intelligence failures, they're context and verification failures.
u/kexnyc · 1 pts
I went through the same “must be magic” phase just like everyone else here. But when the magic quickly degraded, rather than chuck the stinking pile, I regrouped, donned the engineer hat I earned over the last 25 years, and began to look at framing context not prompts. That paradigm shift led me to the same conclusion you outline. But just like any decent agile programmer, I always look for better. I’ll see how the OP ideas stack up.
u/tehsilentwarrior · 2 pts
There’s three things you missed which are very important: - fail fast (which to some extent is akin to fail loud)- no guard clauses- no backwards compatibility Fail fast because you want things to break as soon as possible so they get fixed as soon as possible. Goes hand in hand with no guard clauses. These are your null checks, your default on null, your try catch pass, your if type of whatever return, your x = str(y), etc. You want problems to blow your app out of the water and do it quickly.
u/Joe-Sol · 2 pts
Thanks for sharing! Hiw can I add this to my Cursor ? As a Rule or Skill maybe ?
u/careless25 · 1 pts
CLAUDE.md Or AGENTS.md In the top directory of your codebase
u/AdNecessary1906 · 2 pts
Most of these match what I developed independently over ~200 sessions. A few additions that helped: a START_PROMPT that loads module-specific docs before touching any code — rules tell Claude how to work, docs tell it what exists. And a defined end-of-session doc prompt that updates everything before closing. Next session starts clean but informed. The loop is: START_PROMPT for context, rules for behavior, doc prompt to close.
u/natevoss_dev · 2 pts
The rules-vs-docs split you described is what I've been running, rules in a project-level config file, but "what exists" lives in per-topic files in a memory directory that auto loads at session start. Same loop you described: rules for behavior, docs for existence, close of session updates the docs. One gap I hit there that might be worth adding to your loop: the docs themselves drift. If the close prompt only appends what was done this session, old entries that became wrong don't get corrected. Added a rule that fires when the agent makes a memory derived assumption the user contradicts pause and fix the memory file before continuing, not just at session end. Otherwise the wrong inference compounds across sessions.
u/AdNecessary1906 · 1 pts
Exactly — the drift problem is real. My solution: the closing prompt doesn't append, it rewrites. Every REFERENCE and MAINTENANCE file gets updated to current state, not extended. Decisions and rationale go into a versioned NOTES file that gets archived separately. The full loop: — Session start: START_PROMPT loads base context + module docs + workflow rules — Every task: Assess → Analyze → Build. No step skipped. — Before every build: cross-dependency check — what assumes the old behaviour, what breaks with the new one. Similar to your Verification Gate but anchored to data flow rather than state topology. — Mid-session: 'Stop — check this' as a two-word brake when the data flow gets unclear. Used 200+ times across the project. — Session close: 11-step checklist — version, changelog, roadmap, build manifest, reference docs, maintenance docs, readme variants, tests, notes, next START_PROMPT. Docs rewritten, not extended. Full workflow documented here if useful: github.com/Wewoc/Garmin_Local_Archive/blob/main/docs/MAINTENANCE_GLOBAL.md
u/natevoss_dev · 1 pts
the rewrite not append move covers a different failure than the one I was on. yours catches drift on the way out of a session, mine catches it the moment an inference goes wrong inside one. both belong, end of session rewriting alone misses anything that compounds before close. the "stop, check this" brake is the thing I don't have. mine relies on automated triggers, a manual two word interrupt the user can fire at any point is a different mechanism. curious how you trained yourself to actually use it instead of letting the agent run, that's the part I'd struggle with?
u/elliotttx1111 · 1 pts
helpful
u/mr280890 · 1 pts
Thanks for sharing.
u/vacationcelebration · 1 pts
How does rule 6 work? Does Claude know how much context it has left? And can it start /compact on its own? I'm off the subscription, so I don't know if this was added recently, but at least last time I checked it couldn't start summarizing on its own.
u/FormalAd7367 · 1 pts
If you use an ide like cursor, it will tell you how much it’s got left
u/lifeblunderer · 1 pts
Where are these percentages coming from
u/slightlysublevel · 2 pts
Where do you think? He asked an AI, and that's what Claude told him, so he believed it.
u/Double0Jamo · 2 pts
They’re real OK!….
u/kexnyc · 1 pts
I’ll try it. Always looking to strengthen the guardrails.
u/killerkouki · 1 pts
I’m sorry, but an LLM doesn’t know how to identify assumptions to begin with. All it knows how to do is seek out patterns in a probabilistic manner. So if it got a specific “assumption” wrong then the chances of getting that specific, or variations of, “assumption” wrong again is less likely the next time around. (Yes, I have karpathy’s rules loaded in)
u/LatvianCake · 1 pts
Obviously the LLM hasn't gained consciousness. Stating assumptions forces it to inspect your code and write out boundaries, conditions and edge cases. Because it stated this out loud, it's more likely to stick to it. Like it's generating its own plan.
u/KeenAsGreen · 1 pts
Great ruleset
u/Jonas_Silver · 1 pts
Only if those fucking agents would follow Claude.md
u/Mawuena16 · 1 pts
Right? It's like they have a mind of their own. You'd think following a clear set of rules would be easier, but nope. Gotta keep tweaking those agents.
u/Remarkable-Art1980 · 1 pts
it,s learned on first 4 sure
u/moonman82 · 1 pts
Just trying it …
u/halflucids · 1 pts
If you are getting 41% error rate from claude you have a PEBKAC problem.
u/FutureOfRecords · 1 pts
Ran this against my own error logs from the last two months — about 13 distinct failure classes. The original 4 caught maybe 5 of them. The real leverage was Rule 12 (Fail Loud), Rule 8 (Read Before ▎ Write), and ConferenceMuted4477's claim-grounding rule further down. Numbers look made up; a few of the rules they're stapled to aren't.
u/learner_for_life_11 · 1 pts
Thank you for posting this and thanks to all who have added to this post. It is very helpful. I do have one question though, and it is related to the size of claude.md. In my experience the longer the claude.md file, the more complex it becomes to manage it. I have 2 apps in flight at the moment and the claude.md for each, including some of the best practices and guardrails discussed here, spans 100s of lines. Everytime a build sprint is completed, updting these to reflect the most current state of the development and direction has now become a pain. Any thoughts on how to best address this? I was thinking of breaking up the claude.md into 2 parts, the core claude.md with no more than 100 lines, and moving the rest of the contents to separate reference sections that code can read/refer anytime even during runtime or execution. Has anyone any thoughts on this? Has anyone done this before? Some pointers would be very helpful.
u/hundrethfoo · 1 pts
Try the Linus level skill. A lot of these rules are covered as well as tons of other software engineering standards, security guidelines and a dial for use in different contexts. https://github.com/rsoffer/linus-level-skill
u/ramigb · 1 pts
Does Rule #6 actually work? does claude have access to inferrance engine meta data like tokens count etc? just asking out of genuine curiosity
u/Von__Mackensen · 1 pts
Llms Will find every gap in your rules to cheat. The simplicity and minimal change principles, open even more reasoning gates for the ai to do whatever the fuck its pleases. You tell it to do something more complex and the ai will reason itself out of doing it because it's simpler some other way. Even thou you intentionally gave it a detailed plan. You give the ai quality gates so it can self police and the ai cheats on the quality gates, because it will be a simpler and minimal change. And the better the ai the better they are at reasoning cheating makes sense. I mean, the principle is good and it works. But not as a general claude.md or agents.md rule. Needs to be enforced by skills or sub agents.
u/zlutystrop · 1 pts
Having a spec this wordy can actually set you back. There is most likely a specific reason Karpathy’s original rulebook has so few rules. It isn't that he missed something or suggests you should slap your own rules as needed; it's that the more constraints you specify, the less likely they are to be adhered to, particularly if they are vague, contradictory or ambiguous, which they most certainly are in your case. The fewer rules you pick (=compromise), the more likely the clanker is to honor them. Your "improvements" will actually hinder the model's performance: "Don't continue from a state you can't describe back" Who or what determines what cannot be described anymore? The sycophantic LLM that always tries to please you with an answer? 2. "If you think a convention is harmful, surface it" What makes a convention harmful, specifically? Too wordy? Too blunt? Too permissive? Too strict? How do you even ensure Claude Code is aligned with you on what it means? 3. [At the very top] "Bias: caution over speed" Yet, at the same time, you want Claude Code to adhere to very stringent token limits. So, which one is it?---In your own interest: don't to try act smart if you have no idea what you're doing, as it can verifiably hurt you.
u/bilporti · 1 pts
vibeslop
u/Lucifer38769 · 1 pts
Very helpful. Thanks for this
u/dakamojo · 1 pts
>>The rules worked. Across 30 codebases over 6 weeks, mistake rates dropped from 41% to 11%. How does one test something like this? I would like to do the same. Do you save a point in your code and then have your teams also record every prompt they use? Then reset everything and run the same prompts with the new CLAUDE.md file? How do you count what's a mistake?
u/ProfErber · 1 pts
To start out a codebase don‘t need any of these right?
u/natevoss_dev · 1 pts
Been field-testing the read-before-you-write idea (Rule 8 here) for a couple weeks on r/ClaudeAI. Two gaps worth flagging that keep coming up in comments on those threads: First, Rule 8's inverse failure. Three different commenters across two subs independently named the same thing: agent reads the code, finds the function, builds on top turns out it's a stub or deprecated or never wired to a live entry point. Higher confidence than the "I didn't find it" miss, and harder to catch. u/Stellarton on r/cursor proposed a two-bucket output format that's the cleanest fix I've seen "reachable evidence found" vs "code exists but reachability still unproven" with a quick liveness check before claiming either (what triggers this code, what blocks the trigger, what user path or test actually reaches it). The bucket name does more work than a self-applied confidence label because it's harder to bluff. Second, same session compounding. Once the agent's own earlier wrong assertion enters context, later turns answer against the contaminated context. Rule 10's checkpoint helps but doesn't force re-verification when the user re-tests a prior claim. Worth a separate clause: if the user re-asks any claim from earlier including paraphrased, re-verify from scratch instead of reusing prior turn answers. Also, agree with the comment on Rule 6, Claude can't enforce its own token budget from inside the prompt. That's a runtime concern, the rule reads as aspirational.
u/ArmMysterious896 · 1 pts
Great takeaway mate.
u/Asthmatic_Angel · 1 pts
When your num 1 is think before coding… Jesus Christ
u/SeveralSeat2176 · 1 pts
If you want to try real-world implementation for above: https://github.com/rohitg00/agentmemory
u/Deep_Ad1959 · 1 pts
the 200-line ceiling buried at the bottom is the rule that matters most and almost everyone copying this template is already over it. running the 12 rules plus the project-specific bullets a real codebase needs lands around 250-300 lines for most teams, and the compliance drop past the ceiling cancels most of the gains from the rules themselves. the version i'd actually ship is these 12 pruned down to whichever ones the model violates in your own session logs, then your own project bullets on top. if a rule covers a failure mode you've never hit in your code, deleting it costs nothing and buys budget for one that does fire. the made-up-percentages skeptic upthread is half right by the way, but the ceiling number is the part that holds up in practice.
u/Electronic-Slide-795 · 1 pts
Will try it out! Thanks for sharing.
u/jsgrrchg · 1 pts
In my experience, Claude is misaligned AF. I’m not having these issues with GPT.
u/Fancy_Lecture_4548 · 3 pts
Moving to Codex cut Claude mistakes by 100%.
u/Otherwise_Repeat_294 · 0 pts
you are a genius. Do you know how to code? Are you a coder, I need help all the time my my scripts and stuff

Comments (0)

No comments yet