Discussion on "Code Review Starts with What Must Stay True"

Hiroyuki Nakahata · 2026-10-25T14:30:11.788Z

Software development has long assumed that people will read and understand the code. We choose clear names and break up long functions so that the next person can follow our intent. But when AI produc

Can an ArchMap mark an observation as unknown when the skill cannot work out the order safely? I would want that to stop a Law from passing, rather than let a plausible guess become a checked fact

Thanks for the comment! This has been a core concern since the beginning of ArchSig’s development: we explicitly distinguish an observed zero from something that has not been observed. If an agent cannot determine a value, it should mark it as unobserved, and the tool’s output preserves that distinction too.

django-oscar #4594 reading like a pure lock-ordering fix is exactly why it tests the method well. I keep coming back to the ArchMap step, though. If the model records basket order for the sorted loop, L2 passes for the wrong reason. Who checks the ArchMap against the code before the Laws run, the reviewer or another model?

Thanks for the comment! That’s exactly the issue. I’m currently developing a SKILL to help agents produce more accurate ArchMaps from code. The idea is to improve accuracy by having multiple agents observe the code using that SKILL.

Problems that those observations cannot capture, such as runtime race conditions, are outside the scope of what AAT can establish from the resulting ArchMap.

The stock allocation example is the one that stuck with me. A fix that keeps the count right but shuffles display order looks clean in a diff and still breaks a quiet invariant. Writing those two laws down before the review would have caught it faster than another pass over names. I am going to try one or two must-stay-true lines at the top of my own PRs, even when I am the only reviewer.

Thanks for the comment! I agree. With AI writing code so quickly, I think it’s especially important to spell out the properties we want to preserve and check that they still hold.

Search Hashnode

Code Review Starts with What Must Stay True

Responses6