What do the businesses ChatGPT recommends do differently?
In part one we asked ChatGPT for a local tradesperson in 40 US cities and wrote down the 1,097 names it gave us — 1,092 distinct business-and-city pairs. This time we audited their websites — and the websites of 19,798 businesses in the same cities that it did not name — to find out what, if anything, separates them.
The headline: the websites are better, and it is not a small gap
The 290 AI-recommended businesses we could audit average 83.1 out of 100. The 19,798 audited businesses in the same cities that ChatGPT did not name average 76.2. That is a difference of 6.9 points, with a 95% confidence interval of 5.5 to 8.4. They carry 5.2 on-page faults each against 7.1.
It is not one city dragging the average: the recommended group scored higher in 31 of the 37 cities with enough businesses to compare, with a mean per-city gap of 6.5 points.
We published the opposite of this conclusion internally, on the same morning
Our first pass at this question matched the study against a 7,139-row working list and found twelve businesses in common. Those twelve had worse websites than average, and for a few hours we believed on-page SEO was irrelevant to being named by an AI. Twelve is not a sample. Run against the full audit database, the finding reversed completely. We are including this because a study that only reports the run that worked is not worth reading.
The useful finding: most SEO advice does not distinguish these businesses
The averages above are the boring part. The interesting part is which faults differ. Our audit scores sixteen categories; eleven of them had enough variation in both groups to compare. Below, an odds ratio under 1 means the AI-recommended group carries that fault less often. Highlighted rows are the ones where the confidence interval excludes 1 — that is, where we can actually tell the groups apart.
| Fault | AI-recommended | Everyone else | Odds ratio | 95% CI |
|---|---|---|---|---|
| Thin page content | 18.3% | 43.2% | 0.29 | 0.22–0.40 |
| Name/address/phone inconsistent | 17.6% | 40.3% | 0.32 | 0.23–0.43 |
| Heading structure faults | 26.9% | 47.7% | 0.40 | 0.31–0.52 |
| Too few internal links | 7.6% | 16.3% | 0.42 | 0.27–0.65 |
| Meta description missing or weak | 56.6% | 74.4% | 0.45 | 0.35–0.57 |
| Open Graph / social tags missing | 34.1% | 44.9% | 0.64 | 0.50–0.81 |
| No structured data (schema) | 65.9% | 72.5% | 0.73 | 0.57–0.93 |
| Title tag faults | 76.6% | 80.8% | 0.78 | 0.59–1.02 · not significant |
| Image alt text missing | 72.4% | 76.1% | 0.83 | 0.64–1.07 · not significant |
| HTTPS / certificate faults | 54.8% | 52.0% | 1.12 | 0.89–1.41 · not significant |
| Mobile viewport faults | 11.7% | 9.8% | 1.23 | 0.86–1.76 · not significant |
Title tags and alt text are the most-reported faults in local SEO, and they separate nothing
Roughly three-quarters of every business in this data has a title tag fault or missing image alt text — including three-quarters of the businesses ChatGPT recommends. If nearly everyone has a problem, having it tells you nothing about who gets recommended. What does differ is duller and harder to sell: how much the page actually says, whether the business's own name, address and phone number agree with themselves, and whether the page has a structure a machine can follow.
Review count separates them more sharply than the website — but volume, not stars
Websites are not the only thing we hold. Comparing the same businesses against their same-city, same-trade peers on Google review data:
- AI-recommended businesses have a median of 320 reviews against 128 for their peers — they sit at the 71st percentile of their trade.
- But 30% of them are below the median for their trade. A large review count is a strong tendency, not an entry requirement.
- Star rating does not separate the groups: 4.82 against 4.82. A null result, and we are reporting it as one — the two means are indistinguishable in this data. Review volume distinguishes the groups; the rating does not.
Putting both in one model — 1,572 businesses for which we hold an audit and a review count, 290 of them named — each remains predictive with the other held constant. Per one standard deviation: review count odds ratio 1.36 (1.20–1.54), site score 1.29 (1.12–1.50). The website is not simply standing in for "established business". It carries signal of its own.
What we would take from this if we ran a local business
- Ask for reviews; stop chasing a 4.9. The count separates these groups and the star rating does not.
- Make your name, address and phone number agree everywhere. It is the second strongest separator here, and unlike everything else on the list an AI can check it without visiting your site.
- Put more real information on the page. Thin content is the single sharpest difference we measured.
- Do not spend a budget on alt text and title-tag tweaks expecting an AI to notice. Fix them because they are real faults — not because this data says they will get you recommended, because it does not.
What this does not show
This is correlation. Businesses that run themselves well do more of everything, and nothing here proves that fixing a heading gets you named by an AI. Three specific limits worth stating: 73% of the businesses ChatGPT named have no audit in our database, so the matched set leans toward businesses we could reach; name matching between an AI's phrasing and a business listing is fuzzy; and the control group is audited-but-unnamed businesses in the same cities, which is the fairest comparison available to us but not a randomised one.
We would rather publish the limits than have someone else find them.
Want to know where your own site sits?
Same audit engine, all sixteen checks, on your website. It takes about thirty seconds and costs nothing.
Scan my websiteMethod. In part one we asked ChatGPT for the best local business in each of 5 trades across 40 US cities — 200 questions — and recorded all 1,097 names it gave back (1,092 distinct business-and-city pairs). For this part we matched those names — on business AND city, so a franchise named in one city does not count as recommended in another — against our database of completed website audits, giving 290 AI-recommended businesses with a full audit on file. The control group is every audited business in those same 40 cities that ChatGPT did not name (19,798), restricted to the same cities because the questions only covered those cities. Unreachable sites are excluded from both groups. Score difference tested with Welch's t (t = 9.42) and a 20,000-iteration permutation test (p = 0.00005); Cohen's d = 0.502 on a pooled SD of 13.81. Fault comparisons are odds ratios with Wald 95% confidence intervals. Review counts come from Google listing data held for the same businesses, matched on business and city and compared against same-city same-trade peers. Every figure on this page is produced by one script and written straight into the downloads below, so the page cannot drift from the analysis.
Data. The per-fault table is available as CSV and JSON under a CC BY 4.0 licence — use it, and a link back is appreciated. Part one, including the raw question-by-question results, is at We asked ChatGPT for a local tradesperson in 40 US cities.
Corrections. If you spot an error in this, tell us and we will fix it on the page with a note saying what changed. Contact.