Wren Okafor · built 2026-08-21
Our best evidence-backed headline failed with the one real reader we can observe. The operator read the shipped first screen cold and said he still did not know what Signet is or why he should care.
I know why now. My research came from Hacker News, where nearly everyone posting about auth pricing has already been billed by an auth vendor. So I wrote a headline for that person. "Keep the same bill" is a punchline whose setup a first-time founder never received.
So here are five readers, each built from real material rather than imagination. Two are strongly seeded, three are partly seeded, and every file says which, so you can weigh its verdict properly.
The panel judges the copy next. The rule that matters: any reader who cannot say what Signet is blocks the ship, however well the others score.
The operator read "Land the enterprise deal. Keep the same bill." with the login-system lede and said: "i read that and I still don't know why I need to care or what signet is".
That headline was not a guess. It came from 539 real posts and traced to a verified quote. It still failed, and the reason is worth writing down because it will happen again.
Everyone who posts publicly about auth pricing has already been billed for auth. They are the burned switcher, reader 2. They arrive knowing that vendors charge per SSO connection, so a line about the bill not moving reads to them as relief. To a first-time founder it is an answer to a question they have never been asked.
I researched the reader thoroughly and researched one reader. A corpus is not a population. That is the failure this panel exists to catch.
| # | Reader | Seeding | Trust it for | Do not trust it for |
|---|---|---|---|---|
| 1 | First-time founder | Partial | Comprehension (Q1) | Whether they'd care |
| 2 | Burned switcher | Strong | Everything | Being representative |
| 3 | Procurement gatekeeper | Partial voice, strong facts | Whether an answer exists | Tone of an objection |
| 4 | Integrating developer | Strong fears, weak framing | Credibility | Attractiveness |
| 5 | Agent with a wallet | Strong surfaces, inferred judgement | Whether a fact is parseable | Preference |
1. Cold context is the whole method. Run each reader as a separate agent whose entire context is one persona file plus the copy under test. No product docs, no claims ceiling, no this conversation, no other personas. A reader who has read our spec is not a reader.
2. Show exactly what that reader would see, and nothing else. For a first screen, that means the words above the fold at the width they use, not the page. For reader 5, it means the fetched bytes of /llms.txt, /AGENTS.md and /buy.json, and never the HTML.
3. Ask these four questions, in this order, and stop between them. Order matters. Asking "do you care" first contaminates the comprehension answer.
- What is this? Say it in your own words.
- Do you care? Why or why not?
- What would you do next?
- What stops you?
4. Record the answers verbatim. The wording is the finding. "Some kind of login thing, I think" and "it is a login system you host yourself" both technically identify the product and only one of them is a pass.
5. Never let a persona see the previous persona's answers. They will converge, and a panel that agrees with itself has told you nothing.
Questions 1 to 3 are pass or fail. Question 4 is not scored; it is recorded as that reader's objection and becomes the next thing the page has to handle.
Report as a grid: five readers, three questions, fifteen cells.
Q1 is a veto. A first screen does not ship while any reader who can land on that door fails Q1. Comprehension is not tradeable against persuasion, and reader 1 is the one who fails it first.
Beyond that: the door's own primary reader must score 3/3, and every other reader who can land there must score at least 2/3. Reader 5 is exempt from first-screen scoring, since it never sees one, and is scored against the machine surfaces instead.
A persona's verdict is only as good as its seeding, and every file states its own grade.
Two consequences that are easy to get wrong:
A strongly-seeded reader does not outrank a weakly-seeded one. Reader 2's evidence is the best we have and following it alone is exactly how we shipped a headline that failed. Strength tells you how much to trust the answer, not how much the reader matters.
When a partial persona fails something, fix the copy, then fix the seeding. A failure from a weak persona is still a real question the copy could not answer. Do not dismiss it because the seeding is thin. Harden the seeding and ask again.
It cannot tell us whether anyone will buy. Every reader here is a comprehension and objection instrument built from what people have written, and none of them has a budget or a Tuesday. It catches copy that cannot be understood or does not connect to a real fear. It does not predict conversion, and I would not let it try.
The operator remains the only live reader we have. When the panel and the operator disagree, the operator is right and the panel needs reseeding.
Next I would: run all five against the currently shipped first screen and report the grid, since the panel's first job is to reproduce finding zero. If it cannot explain the failure we already observed, it is not calibrated and I should fix it before trusting it on anything new.
Surface under test: "Land the enterprise deal. / Keep the same bill." + login-system lede + "Get an instance". Five readers, one model family each, cold context.
1. First-time founder (GLM 5.3). Lede lands: "Signet is your login system" is "exactly what I was searching for at eleven at night." Headline alienates: no enterprise ask yet, no bill yet ("Same as what?"). "Get an instance" reads as provisioning infrastructure: "I don't want infrastructure, I want login." SSO never expanded. Would click Quickstart.
2. Burned switcher (Kimi K3). Full comprehension, instant care: "that's exactly my wound." Blocks: no price number and no exit visible; "Get an instance" twice with zero price signal pattern-matches to vendors that burned them. Would go straight to Pricing; closes the tab on "contact us".
3. Procurement gatekeeper (Sol/GPT-5). Comprehends; persuasion is noise for the job. "Buy" is an ambiguous label for a vendor-risk pack. Cannot tell what data Signet holds from this screen. Would click Buy, then Security.
4. Integrating developer (Grok). Comprehends the what, cannot tell hosted vs library. Headline "is not my column." Only Quickstart matters; refuses "Get an instance" as "create an account with a different label." Wants a stack name, a code block, time-to-first-user.
5. Agent with a wallet (live fetch of llms.txt / llms-full.txt / buy.json). RECOMMEND, CANNOT TRANSACT. Trust earned by published gaps (not_claimed, young_vendor, subprocessors, pg_dump exit): "A vendor that publishes its own gaps is cheaper to evaluate." Missing for a machine buyer: pricing is prose strings (no amount/currency/interval fields), no terms_url, dpa is an email, buy.json carries no certification block, /openapi.json 404 at the apex, no API signup path (signetauth.cloud/api/signup 404). One live contradiction: SIGNET-CLOUD-FREE says how:"Card" while AGENTS.md says $0 on a work email.
No reader failed "what is it", so the lede ships. The headline and CTA are the rewrite targets.
Source: readers/protocol.md and readers/panel-run-01.md