Our menu extractor put "not vegetarian" in the field for diets a dish is suitable for

a beef burger gets no V tag at all

# The Beef Burger With No V Tag

A beef burger gets no V tag at all. That sounds like the correct answer, and it is. The trouble was that nobody could tell whether it was correct for the right reasons, which is a much worse place to be than getting it plainly wrong.

Symptom

The symptom, exactly as it showed up in the london-menus output: a beef burger gets no V tag at all. Fine. Good. That's what you want.

But the tags around it were a mess. Dishes that should have carried a vegetarian marker came back without one, while other dishes picked up claims that didn't belong to them. I mean, it wasn't random, but it wasn't predictable either, and that's the part that bothered me. A bug you can reproduce is a bug you can fix. This one only reproduced on some dishes, depending on how the model happened to phrase its answer.

So the burger was the clean example. It's the case where the pipeline did the right thing, and I kept coming back to it because it showed that "right output" and "working system" were two different things.

What we thought it was

First guess: the model was just being sloppy. Models hallucinate, you know, and menu text is thin. A line like "burger, bun, relish" gives very little to reason from, so a wobbly inference felt completely believable.

Second guess, which followed from the first: tighten the prompt. Tell the model to be more careful, more conservative, to think harder about meat. I spent a while on that. It felt productive, actually, because the outputs did shift a bit with each rewording.

That was the trap. Prompt tweaks move the output around, so you always get a little signal back, and a little signal feels like progress. Anyway, the shifting wasn't fixing anything. It was just reshuffling which dishes failed.

The wrong diagnosis was believable for a boring reason: the model really is sometimes imprecise. It just wasn't the cause here.

Root cause

The real problem was in how we asked the question. The extraction prompt, at tools/extract-prompt.md:22@a3502bf^, requested a single field called diet_inferred. One field, holding "what it inferred."

And what it inferred was two completely different kinds of statement. Some were positive claims, like suitable for vegetarian. Others were negative ones, like contains meat or has nuts. Same field, same shape, opposite meanings.

Think about what that does downstream. A tag of "vegetarian" says the dish is safe for that diet. A tag of "nuts" says the opposite, that you should be careful. The field didn't record which was which. It just held strings.

So we had a second stage trying to clean up after the first. The normalisation script, at tools/normalise.mjs:11@a3502bf^ and again at tools/normalise.mjs:28@a3502bf^, ran a regex pass over the tags. Its job was to work out the polarity of each one by guessing from the tag text.

Guessing polarity from text. Right. That's the whole bug in one sentence.

A regex can match "meat" and decide it's a warning. It can match "vegetarian" and decide it's a claim. But the model's phrasing wasn't fixed. "No meat", "meat-free", "contains meat", "vegetarian-friendly": these all sit near each other in the text, and they don't all mean the same thing. The regex was inferring structure that the data had thrown away upstream. Once two meanings collapse into one field, no amount of pattern matching gets them back reliably.

This also explains why my prompt rewrites went nowhere. I was adjusting the wording of an ambiguous answer, when the ambiguity was baked into the shape of the answer itself.

The fix

The commit, a3502bf, landed on 2026-10-21, and it did the obvious thing once the problem was clear: it split the field in two.

suitable_for now holds only diets a dish qualifies for. It never holds a warning. contains holds the things in the dish. The two never mix, so there's no polarity left to guess.

The safety pass changed role as well. It used to repair tags. Now it only removes diet claims that clash with the dish text, for example V on a dish with meat in it. It never adds one. That asymmetry matters. A pass that can only subtract can make the output more cautious, but it can't invent a claim that wasn't there.

Two more changes rode along in the same commit. Gluten-free and nut-free inferences are capped at low confidence, because a wrong "safe" claim there is the expensive kind of wrong. And halal is no longer inferred at all. You can't read it off a menu line, so we stopped pretending to.

Why it happened

Here's the systemic part, and it's less about this one prompt than about how it survived review.

A single field with a friendly name looks fine in a diff. diet_inferred reads naturally. Nobody reviewing the prompt line sees a problem, because the problem isn't in any line. It's in what the field is allowed to mean. And the regex pass looked like diligence, like somebody had already thought about the edge cases. It actually papered over the design flaw, which made the flaw harder to see.

That's the thing about repair code. It makes a bad contract look like a working one. Reviewers read the cleanup, see that it handles the cases, and move on.

How to avoid it

If you're extracting structured data from a model, give every field exactly one meaning. If a value can be a claim or a warning, those are two fields. Ask for them separately.

Be suspicious of any step whose job is guessing what an earlier step meant. That's a sign the information was lost upstream, and you can't fix it downstream.

And make your safety checks subtractive. Let them remove risky claims, never add reassuring ones.

Anyway, the burger still gets no V tag. Now it's for the right reason.