The proposal writer that isn’t allowed to guess

When someone tells us about a job, on WhatsApp or in the chat on our site, they can ask for a proposal. A model writes it: a headline in their terms, why the job is worth doing, what we’d do on the first day, in the first week and over the first three months, and two or three questions we’d ask on a 30-minute call. It lands on a private page they may forward to a business partner, so it has to make sense on its own.

The prompt carries one rule we care about more than the rest: “Never write a figure they didn’t give you: no amounts, percentages, hours, days or time saved. Their own numbers are fine.” A proposal is written before we’ve seen the job. Any saving it promises is a promise we haven’t earned.

On 14 October we read the first six live proposals side by side. Some of what we found was easy to fix. The page quoted the person’s last message instead of their job. The model picked “prototype” as the service for four of the six, including a padel venue that wanted a website. Two subheads ended in stray characters.

The worst of it was the figures. Despite the rule, sentences like “saving you hours each week” were getting through. None of those prospects had given us a figure like that. The model was writing the kind of sentence proposals usually contain, which is exactly the problem. A rule in a prompt is a strong suggestion. If something matters, the code has to check it.

So every proposal is now checked before anyone sees it. inventedFigures() splits everything on the page into sentences and flags any that:

• has a £ sign, a % or “per cent”, when the person never used one;
• has a number that isn’t in what they told us (a year is fine);
• claims time saved: “save”, “saving” or “free up” within a few words of hours, days, minutes or a number, or hours “back”.

Their own numbers pass. If someone says they post five articles a week at 20 minutes each, the proposal can say so. Numbers written as words aren’t checked, on purpose, so a sum on their own figures (“roughly an hour and forty minutes a week”) stands.

writeProposal() does the rest (commit 2a9ee26, 14 October). A clean first draft is one call and we’re done. A flagged one is sent back once, with the offending sentences quoted and a note that begins “A first draft of this page stated figures nobody gave us, which we never do”. If the second draft is still flagged, or the rewrite fails, the flagged sentences are cut out. A field left empty isn’t shown.

One rewrite, not a loop. The extra call only happens when the first draft invents something, so honest drafts cost nothing more. Cutting a sentence costs a little polish. Leaving it in costs the trust the page is there to earn.

The first version of the scrub had faults, and they were fixed the same night (40a38c6). It flagged our own offer: the page proposes a 30-minute call, and “30” usually isn’t in what they said. It flagged questions like “Which jobs would free up the most hours?”, which claim nothing; a question about time saved now passes, and a statement still doesn’t. Cutting a sentence out of bold text left stray asterisks behind. And a flagged headline repeated inside a longer flagged sentence could leave half that sentence on the page, so the longest sentences are now cut first.

Once the code holds the line, the risky moment is a change to the prompt. So that has its own check. php craft mud/proposal-check (1d012a3) writes six made-up proposals, shaped like those first six live ones, through the same call the website chat and WhatsApp both use. For each, it prints the service the model chose against the right one, the call questions, and any sentence that still states a figure nobody gave us.

It fails if a figure gets through or any of the six lands on the wrong service; the padel venue and the two WordPress jobs are the ones the old prompt got wrong. It costs about $1–2 a run, and nothing is saved or sent to anyone. Our server notes say to run it after any change to the proposal prompt.

Put the rule in the prompt, because it helps. Then check it in code, because the prompt alone didn’t hold. Give the model one chance to fix its own work, and tell it exactly which sentences were wrong. When it can’t, take the claim out rather than ship it. And keep a small set of real-shaped cases you can replay, so the next change to the prompt is tested against the last mistakes.