An AI reviews each offer developers make to a group of home buyers. When it invents a discount, its review is thrown away.

On Groupliv, people who want to buy a new apartment form a group, and developers send that group private offers. The members compare them and vote. When an offer arrives, Liv, the product’s AI advisor, writes one review into the group’s feed: is it cheap or expensive for the area, and what is weak about it. Every member who opens it reads the same review.11Checked 1 Nov 2026: 11 developer offers published to 3 groups with real buyers since 11 Aug 2026; all 11 carry an AI review

One mistake there is worse than all the others: a false sentence about money. Writing “after the discount” about an offer with no discount misleads a whole group about the thing it is negotiating. Telling the model not to was the obvious fix. It held on one run and broke on the next, from the same facts.2221 Aug 2026: from identical facts, one run said 'no discount', the next 'after the discount'. The check went in that day: 10 phrases, 4 tests

The calls I made

The AI writes sentences; code does the sums. I could have handed the model the prices and the area’s market data and let it compare. The first version did roughly that, without the offer’s floor area. Given a benchmark and nothing to divide, it said nothing about price at all.3312 Aug 2026: given an area price per m² and no figure for the offer, the AI said nothing on price. Now code computes it; under 20 deals, no benchmark Now the price comparison arrives finished, computed in code, and the same figure sits on the offer card on its own, so a buyer sees it whether or not the AI mentions it.

A check in code, not firmer wording. When the AI described a list price as discounted, the tempting fix was a stronger instruction. But the same facts produced the right sentence on one run and the wrong one on the next. Wording can lower the odds of a sentence like that; it cannot remove them. So when an offer has no discount and code has priced it, code reads the review and refuses it if it uses any of ten phrases that present the price as net of a discount. The list is narrow on purpose: a review that says the offer has no discount must pass, or the retry would fail the same way.

An empty slot over a wrong sentence. A review that fails the check is regenerated once. If the retry says it again, the review is dropped instead of posted. I chose a gap over a guess: the card still shows the measured price, and the group can still read the offer itself. A false claim about a discount, once members have read it, cannot be taken back.

What every member reads gets only what every member may see. On 29 September a one-member group’s review quoted that member’s budget back to the group. The AI stopped receiving budgets that day.4429 Sep 2026: a one-member group's review quoted that member's budget. Budgets left the AI's facts that day; a test checks both languages A review is a shared document, so personal numbers stay out of it, even when they would make it sharper.

What happened

Eleven offers have reached groups with real buyers, and all eleven carry a review.11Checked 1 Nov 2026: 11 developer offers published to 3 groups with real buyers since 11 Aug 2026; all 11 carry an AI review Five of those reviews were of offers with no discount. All five were written on 19 and 20 August, before the check existed, and every one of them contains a phrase it now refuses.55Checked 1 Nov 2026: all 5 reviews of offers with no discount date from 19–20 Aug and hold a phrase the check now refuses; the 4 written since are of discounted offers The four reviews written since were all of offers that did carry a discount, so the check has not yet had to fire in production. It was built for the next offer without one.

Under the hood

The AI here is a single structured call that returns a review and a confidence. It has no tools. Everything it knows is gathered before the call and rendered into a block of facts appended to the prompt.

The numbers. Code loads the apartments the offer covers and computes their price per square meter net of the offer’s discount, as a median so that one penthouse in the set barely moves it. The benchmark is a median of recent new-build deals in the same neighborhood or city, quoted only when at least 20 deals stand behind it; below that the facts say there is no reliable benchmark and tell the model not to rule on cheap or expensive.3312 Aug 2026: given an area price per m² and no figure for the offer, the AI said nothing on price. Now code computes it; under 20 deals, no benchmark Every line of the facts that carries the gap names its scope, city or neighborhood, because a city median read as “the area” is a different claim.

The failure. For an offer with no discount, the facts block carries the developer’s list price unchanged. On 21 August the facts block gained an explicit line for that case: this is the list price, do not describe it as after a discount. Measured the same day, with that line in place, on an offer with no discount: one generation said the offer carried no group discount; the next, from identical facts, wrote “after the discount from the developer’s price list”.2221 Aug 2026: from identical facts, one run said 'no discount', the next 'after the discount'. The check went in that day: 10 phrases, 4 tests Those facts were not clean. An older line, written for offers that do carry a discount, still went out with every offer: the discount is off the developer’s list price, and the actual price is lower. The model was holding two instructions that pull apart, and on 21 August it followed each one once. That line should only appear when there is a discount. But the check does not depend on the prompt being right: the next contradiction will not announce itself.

So the check went in the same day, right after the model answers. This is the code as it runs:

if _claims_a_discount_that_does_not_exist(result.output.verdict, ctx):
    logger.warning("advisor_claimed_absent_discount", offer_id=str(offer_id), attempt=1)
    result = await agent.run(
        prompt,
        deps=ctx,
        usage_limits=UsageLimits(request_limit=settings.agent_max_request_limit),
    )
    if _claims_a_discount_that_does_not_exist(result.output.verdict, ctx):
        logger.error("advisor_verdict_dropped", offer_id=str(offer_id))
        return None

Narrow on purpose. The check fires only when the offer has no discount, and only on ten phrases, six Hebrew and four English, that present a price as net of one: “after the discount”, “before the discount”, “post-discount” and their kin.2221 Aug 2026: from identical facts, one run said 'no discount', the next 'after the discount'. The check went in that day: 10 phrases, 4 tests The word “discount” alone is not on the list. A review saying “the offer includes no discount, and the group should ask for one” is correct and must pass, or the retry would fail the same way and every honest review of such an offer would be dropped. A false positive costs a regeneration, at worst the review, never a wrong number.

Four tests pin the check: a false claim is caught in both languages, naming the absence passes, an offer with a discount may describe it freely, and an offer with no computed price is left alone.2221 Aug 2026: from identical facts, one run said 'no discount', the next 'after the discount'. The check went in that day: 10 phrases, 4 tests Nothing re-runs a dropped review: the drop is logged as an error, and the group sees no review for that offer. The retry-and-drop around the check has no test of its own yet. That is the gap I would close next.

What transfers

• Give the model finished numbers. If it has to do the division, it can get it wrong; here it said nothing.
• A prompt instruction is a request. For the one false statement you cannot afford, put a check in code on every output, and keep it narrow enough that the honest answer passes.
• Prefer a visible gap to a confident error. Drop the output, and keep the measured number on screen on its own, where it never depended on the AI.