The 1950 children's book Petunia by Roger Duvoisin tells the tale of a goose who finds a book in a meadow. She clutches the book under her wing and proudly struts around the farm for all the other animals to see how smart she is. Puffed up with false confidence, she doles out terrible advice to the other animals.

Relying on AI to assert code quality is like carrying a book around to look smart. It goes through the motions, but there is no substance.

GenAI is fast but sloppy

In 2026, many professional software developers, not just project managers and car mechanics, have become convinced that it's OK to skip human code review. I can think of at least two beliefs that support such a conclusion:

1. Internal code quality doesn't matter as much as shipping now.
2. Today's LLMs are adequate for ensuring code quality.

In my last post, I typed 3200 words investigating the first belief. In short, sometimes internal code quality doesn't matter, like for hobby projects or internal software, but in business, the organization's appetite for quality hinges on its priorities. These priorities always reduce to some variation of "make more money" and "spend less money". Except for the most reckless or desperate of organizations, if the project is valuable enough to justify compensating a professional software engineer, internal quality matters.

Let's now turn our attention to the second belief. The book xUnit Test Patterns¹ was published on January 1, 2007. It's been out for twenty years. Thanks in part to this book and its blessed author Gerard Meszaros, we've known for at least two decades what test smells are, which design choices make software hard to verify, and which quality control choices make software resistant to change and therefore expensive to maintain. Nobody's debating these principles.

Yet generative AI seems not to know about them. In a recent pull request at work, Claude generated the code below, and it passed multiple rounds of adversarial code review. I invoked ten review agents, a mix of Claude and Codex. I didn't provide them any extra knowledge of good test practices. None of the agents pointed out the deficiency I am about to reveal to you.

Avoid behavior verification

Take a look at this code.

class FakeCache<T> : ICache<T>
{
    public int FetchCallCount { get; private set; }

    public T Fetch(int id)
    {
        FetchCallCount++;
        return default;
    }
}

[Fact]
public void GetById_DoesNotUseTheCache()
{
    // Arrange
    var cache = new FakeCache<Widget>();
    var service = new WidgetService(new FakeClient(), cache);

    // Act
    service.GetById(id: 1);

    // Assert
    cache.FetchCallCount.ShouldBe(0);
}

What's wrong with this test? It's asserting an interaction where it should be asserting a contract.

    cache.FetchCallCount.ShouldBe(0);   // asserts an interaction

The public contract of GetById is that it always returns fresh data, but what we've tested here is whether it calls Fetch. This is also called behavior verification² because it verifies what the code does rather than what it achieves.

Imagine some day Big Boss slinks up to your desk and says, "The dashboard's too slow. It's driving customers nuts. Make it load faster." You instrument, trace the latency to caching, and implement the obvious fixes, and in doing so you change how many times Fetch gets called. Every test that asserts on that fact turns red. Now you have to fix those tests. You can avoid that extra maintenance burden. The public contract hasn't changed, so neither should the tests.

The disadvantages of behavior verification are severe:

• It makes the test brittle. It relies on facts that happen to be true right now but are not guaranteed to remain true.
• It makes the code under test resist change. Whether you want to rename Fetch, add a parameter, or make it async, every test that asserts on it turns red.

As Mark Seemann wrote four years before ChatGPT³:

The experience that most people seem to have, though, is that when they change something in the code, tests break. This is a well-known test smell. In xUnit Test Patterns this is called Fragile Test, and it's often caused by Overspecified Software....the tests are highly coupled to implementation details of the software. The cause is often that...test verification hinges on how the System Under Test (SUT) interacts with its dependencies.

Prefer state verification

I prefer to rely on guarantees, not hopes and dreams. That's why I always reach first for state verification. It relies on public contracts, not private implementation details, which compels you to design modular code.

Sometimes there isn't any choice and we have to resort to interaction testing, but in this case, there's a natural state we can verify: the return value of GetById. If it returns a stale value, the test fails.

Here's a redesigned FakeCache:

class FakeCache<T> : ICache<T>
{
    private readonly Dictionary<int, T> _entries = new();

    public void Set(int id, T value) => _entries[id] = value;

    public T Fetch(int id) => _entries.TryGetValue(id, out T value) 
        ? value 
        : default;
}

Now after calling GetById, instead of counting invocations, the test asserts what the SUT returns:

[Fact]
public void GetById_ReturnsFreshData()
{
    // Arrange
    int id = 1;
    var cache = new FakeCache<Widget>();
    cache.Set(id, new Widget("Stale"));

    var service = new WidgetService(
        new FakeClient { FreshResponse = new Widget("Target") }, 
        cache
    );

    // Act
    Widget result = service.GetById(id);

    // Assert
    // We'd see "Stale" if the data was not fresh
    result.Name.ShouldBe("Target");   
}

Now when Big Boss walks up, you can chuckle confidently knowing your tests have got your back.

What to do now

When you go to a restaurant, you don't care whether the chef cooked your meal in one pan or in five. You care that the food is delicious.

By the same spirit, avoid behavior verification. Run away from code that resembles Assert(Thing.WasCalled(Times.Once)).

Instead, prefer state verification. Stick to public contracts. Assert on return values or mutated state. Let what other functions your function calls remain a secret to your function.

Now that you know the difference, you can choose. Beware though, you have to be vigilant. I find Claude often tries to sneak in behavior verification.

For Petunia, things start to go well again when she actually opens the book and reads it. Your reward for exercising good test discipline is the delight of software that is easy to change.

References

1. xUnit Test Patterns
2. Behavior Verification
3. From interaction-based to state-based testing

Image Credits

Petunia is a character created by Roger Duvoisin in Petunia (Alfred A. Knopf, 1950). Illustrations © Roger Duvoisin. The Knopf imprint is now part of Penguin Random House.

The book cover is from xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros (Addison-Wesley, 2007). Cover © Pearson Education, Inc.; Addison-Wesley is an imprint of Pearson.

Both images were edited and upscaled with AI.