Executive summary (the part you can read in 60 seconds)
- When a chatbot “doesn’t know your company,” the problem is often not awareness. It’s name matching. Small changes in punctuation, abbreviations, or legal naming can make an entity look like a different thing.
- In the LLMtel 1,000-entity benchmark (17 LLMs per entity), the same organization written two slightly different ways lost 1 – 3 models of recognition. That’s a 6%–18% drop from nothing more than formatting.
- Wikipedia’s real superpower is not fame. It’s canonicalization: a stable page title plus redirects that collapse messy name variants into one identity. For AI training data, that behaves like an entity identity resolver.
Lead: “Why don’t chatbots know us?”
Picture this: your team rolls out an AI assistant on your website. A customer asks about your company. The bot answers like you don’t exist.
So you test it yourself. You type your company name.
Nothing.
Then someone tries a slightly different version maybe with “Inc.”, maybe without parentheses, maybe the old brand name, maybe the domain name.
Suddenly the bot “knows” you.
That’s the twist: the model may know your organisation, but not the exact string you typed.
This is a canonical-name failure:
Small string variations → big recognition swings.
What we measured (and why this dataset is different)
This comes from the LLMtel benchmark, which tested “AI visibility” at scale:
- 1,000 entities
- 17 chatbots/LLMs queried per entity.
- 5,070 total prompts
- 86,190 answers
The study uses two scores. They sound similar, but they measure different things.
Metric 1: Entity Score “Does the AI know this name?”
This score answers one question:
“If we directly ask each AI about this entity, how many of them recognize it?”
We tested 17 different LLMs, so the score is always out of 17.
- Entity_num = how many of the 17 models recognized the entity
- Entity Score = Entity_num / 17
- Entity_pct is the same number written as a percentage
Example:
If 14 out of 17 models know the entity:
- Entity Score = 14/17
- Entity_pct = 14 ÷ 17 ≈ 0.8235 (82.35%)
This tells you how familiar the models are with the entity when asked directly.
Metric 2: Questions Score “Does the AI mention this name on its own?”
This score answers a different question:
“When we ask many questions across many models, how often does this entity show up in the answers?”
The math:
- Q_num = number of times the entity is mentioned in answers
- Q_den = total number of answers we checked for that entity
- Usually this is 17 × the number of questions we asked, but sometimes it’s smaller (for example, if a model skipped a question due to safety or policy).
Then:
- Questions Score = Q_num / Q_den
- Q_pct is the same number as a percentage
Example:
If an entity appears 74 times in 170 possible answers:
- Q_pct = 74 ÷ 170 ≈ 0.4353 (43.53%)
This tells you how likely the AI is to bring up the entity by itself in real conversation.
The canonical-name failure: what it is, and why it breaks AI visibility
Canonicalization means: turning a messy real-world name into one stable identity.
In practice, it’s normalizing things like:
- capitalization (“NVIDIA” vs “Nvidia”)
- punctuation (commas, apostrophes)
- parentheses (“(CMA)”)
- suffixes (“Inc.”, “Ltd.”)
- abbreviations vs full names (“PRSA” vs “Public Relations Society of America”)
Why it matters
If your identity is split across variants:
- Your “AI awareness” gets fragmented
- LLMs may treat variants like different entities
- Or they may fail to match a variant to the “known” form
Common failure modes (seen constantly in real life)
- Missing/extra parentheses
- Abbreviation vs full name
- Legal entity name vs brand name
- Domain-only form (example.com)
- Rebrands and legacy names
- Minor typos and spacing differences
The proprietary proof: normalization pairs that split signal
The dataset includes normalization pairs: two surface-name variants that humans clearly understand as the same entity, but models treat differently.
Here’s the simplest way to quantify the damage:
Variant Penalty (the math)
For any two variants of the same entity:
- Entity_pct = Entity_num / 17
- Variant penalty (models lost) = Winner_Entity_num − Loser_Entity_num
- Variant penalty (Entity_pct) = (Winner_Entity_num − Loser_Entity_num) / 17
So losing 3 models is:
- 3 / 17 = 0.1765 → a 17.65 percentage point drop in recognition.
Real examples from the benchmark
A) PRSA variant penalty (punctuation broke recognition)
- Variant A: “Public Relations Society of America (PRSA)” → 17/17
- Variant B: “Public Relations Society of America (PRSA” → 14/17
Math:
- Models lost = 17 − 14 = 3
- Entity_pct drop = 3 / 17 = 0.1765 (≈ 17.65%)
Interpretation: one missing “)” corresponded to a 3-model recognition drop.
B) CMA variant penalty (formatting moved the needle)
- Variant A: “Canadian Marketing Association (CMA” → 16/17
- Variant B: “Canadian Marketing Association (CMA)” → 14/17
Math:
- Models lost = 16 − 14 = 2
- Entity_pct drop = 2 / 17 = 0.1176 (≈ 11.76%)
Interpretation: one missing “)” corresponded to a 3-model recognition drop.
C) Smaller brands show the same pattern
This is not only a “big brand” phenomenon.
- Fulton Roark: 3/17 → 5/17 depending on spacing/format
- Models gained/lost = 2
- Entity_pct swing = 2/17 = 0.1176
- Matt Steve’s:4/17 → 5/17
- Models gained/lost = 1
- Entity_pct swing = 1/17 = 0.0588
Why Wikipedia is the stabilizer(the “entity identity resolver” argument)
Here’s the point most people miss:
Wikipedia doesn’t just describe entities. It standardizes them.
Think of Wikipedia like the DNS system for names:
- Humans type messy strings.
- DNS resolves them to one destination.
- Wikipedia resolves messy organization names to one canonical entity.
Mechanisms that matter
- Canonical page title
The title becomes the “default label” copied across the web. - Redirects
Redirects collapse variants (abbreviations, old names, punctuation differences) into one identity. - Disambiguation pages
For short or generic names, disambiguation reduces confusion (“this one, not that one”). - Infobox + structured fields
A consistent structure reinforces the same entity attributes over and over. - Wikidata linkage
Often connected: a stable ID plus external identifiers.
Why that helps LLMs
LLMs learn patterns from huge text corpora. When the same entity is repeatedly presented with
- the same name string,
- the same linked references,
- the same attributes,
…the model has a much easier time “grounding” the name.
Redirects matter because they teach:
“These different strings point to the same entity.”
That is exactly what your normalization pairs show models struggle with when that resolver layer is missing.
Why Wikipedia is the stabilizer(the “entity identity resolver” argument)
Even if a model vendor says “we don’t train directly on Wikipedia,” Wikipedia-derived content is:
- widely mirrored,
- heavily referenced,
- repeatedly quoted and summarized,
- embedded into many secondary datasets.
So Wikipedia’s naming decisions leak into the wider training environment.
Note:
Entity recognition improves when the model sees repeated co-occurrences:
- (name) + (industry) + (location) + (key facts)
Redirects increase how often those co-occurrences happen across variants.
The bigger pattern: visibility is a pipeline (Known vs Named)
Canonical naming is the identity layer. But visibility also depends on question intent and context.
The 1,000-entity study shows two big buckets:
Known but never named
- 319 entities were recognized by at least one model (Entity>0) but never appeared in answers (Q=0).
- As a share of the dataset:
319 / 1000 = 0.319 = 31.9%
What this means
- Canonicalization helps models map you correctly (recognition).
- But recommendation depends on:
- the question asked,
- the category the model thinks you belong to,
- whether the model feels safe naming you as an answer.
Wikipedia helps with identity. It doesn’t guarantee recommendation.
Practical implications for executives
If you run a brand or organization
Treat naming consistency like infrastructure, not marketing copy.
If your identity is split across variants:
- AI search and chat will undercount you
- buyers will not see you
- customer support bots will fail to reference you
- partners and press will amplify different name forms, making the split worse
If you use AI visibility rankings
Be careful: “AI awareness” metrics can be wrong if the scoring doesn’t normalize variants.
- A leaderboard might punish you for:
- punctuation,
- parentheses,
- abbreviations,
- old names from before a rebrand.
Wikipedia helps with identity. It doesn’t guarantee recommendation.
Playbook: how to stabilize your name for AI (ethically)
Step 1 – Run a “variant audit”
List every variant people use:
- legal entity name
- brand name
- abbreviations
- domain forms
- old names
- punctuation variants
Step 2 – Pick a canonical
Choose one “primary” display name and lock it:
- Exact spelling
- Exact punctuation
- Consistent spacing
Use it everywhere:
- homepage
- About page
- press boilerplate
- partner pages
- social bios
- executive bios
Step 3 – Strengthen entity resolution signals
- Add structured organization metadata (schema.org Organization)
- Create a consistent “About” page that repeats the canonical name
- Ensure high-authority citations repeat the same string
If eligible and appropriate:
- Wikipedia + Wikidata can serve as a powerful public identity resolver
but do this ethically.- follow notability rules,
- rely on independent sources,
- handle conflicts of interest properly.
Step 4 – Monitor drift
Rebrands, product lines, and global naming differences create new variants. Track them quarterly.
Conclusion
AI doesn’t “forget” you. It often fails to map you.
Wikipedia’s superpower is not fame. It is canonicalization:
- one stable title,
- many redirects,
- fewer identity splits.
If you care about being findable in LLM answers, treat naming consistency like infrastructure.