By Neycho Tepavicharov, Co-founder, Flashcloud
We have largely accepted that AI assistants get things wrong. Such tolerance is reasonable enough when the stakes are low. A slightly stale restaurant opening time costs someone a wasted trip. A wrong product spec gets corrected at checkout. The error is annoying and self-limiting, but that’s pretty much where it ends.
To be specific about our scope here, this is not about medical advice. That is a separate problem for others more qualified to parse.
This is about the mundane operational facts of a practice – where it is, what it treats, which providers work there, whether it is taking new patients, how you make an appointment. Ordinary directory information, of the kind every business has.
Those facts get read, summarized and repeated by AI assistants the same way restaurant hours do. The difference is what happens when they are wrong.
The Stakes Are Not Evenly Distributed
Someone looking for a healthcare specialist, for example, is usually researching under time pressure, often in an unfamiliar area, and frequently at a moment when they are least equipped to verify what they are told. They ask for a clinic that treats a particular condition and accepts new patients, and they get back two or three names with a line about each.
If the specialty listed is one the practice stopped offering two years ago, the patient calls and, ultimately, gets turned away. If the location is a site the practice closed, they travel to it. If the named provider left, they ask for someone who is not there. None of these are catastrophic in isolation. All of them are worse than the equivalent error about a restaurant, and they happen to people who are already having a bad week.
There is a second-order effect worth noting too. Wrong information about a medical practice tends to be repeated with the same confident tone as correct information, and the person receiving it has no obvious way to assess which they have. The usual signal a human reader would use – a website that visibly has not been touched in years – is stripped out entirely by the time it reaches a summary.
Nobody Involved Can See It Happening
This is the part that makes it hard to fix rather than merely an unfortunate reality.
Practices have almost always learned of wrong information through complaints. Someone turns up at the closed office and rings to say so. But that loop requires the affected person to make contact.
An AI-mediated error can break that loop because the decision may happen before the patient reaches the practice. If the AI assistant leaves the practice out, describes it poorly, points to another provider with clearer information or relies on an outdated listing, the patient may simply move on. The practice never gets a visit, call or form submission, so there is nothing to register. The missed opportunity disappears before it becomes data.
I run a hosting company, so I see a lot of small business websites from the server side, including plenty of medical and dental practices. The pattern is consistent and it’s not due to being careless. A practice website tends to sit between the office manager, whoever built it years ago, and nobody. It gets updated when something forces the issue. Meanwhile it often falls under the radar and becomes the primary source AI uses to describe the practice to prospective patients.
Some practices do not have a working site at all, which is its own version of the same problem – the assistant assembles a description from whatever third-party listings it can find, and the practice has no input into any of it.
Why Healthcare Exposes The General Failure
Healthcare matters here because it makes the larger problem easier to see. AI does not have to invent a wrong answer to cause harm. It can simply repeat outdated information from a source the organization forgot to maintain.
The failure is not that models hallucinate, though they do. It is that they faithfully report stale source material, with no mechanism for signalling age of the information, or confidence that it’s correct, to people who have no way to check. The model is working as designed. The input was wrong, and nothing in the pipeline flags that.
That applies everywhere. Healthcare just makes it legible, because the consequences are concrete enough to notice and the information involved is unambiguously factual. There is no interpretation involved in whether a clinic has moved.
It also suggests where the fix has to sit. Improving the model does not help if the source is wrong. The correction has to happen at the source, which means the organisations being described have to know they are being described, and currently most of them do not.
What That Means Practically
For anyone running or advising a business in a category where accuracy matters, the immediate step is as simple as finding out what is being said.
This takes about twenty minutes. Ask different AI assistants the questions a customer would ask, in the words a customer would use, and note whether you appear and whether the description is right. We wrote up how to do that properly, including which sources to look at once you have the answers, because the sources cited are usually more useful than the answers themselves.
Then assign it to someone. Most of these problems persist not because they are hard but because no individual is responsible for whether the public description of the organization is accurate. That responsibility tends to live as a vague assumption spread across marketing, operations and whoever built the site.
And check that the site actually responds. Systems gathering information move fast across many sources. A site that is slow or briefly unreachable when visited is skipped silently, and whatever less accurate source responded gets used instead. A practice can maintain a scrupulously current website and still be misrepresented because it was unavailable at the wrong moment.
A Reasonable Expectation
None of this is an argument against AI assistants in high-stakes categories. They are useful, people are already using them, and that is not reversing.
It is an argument that the tolerance we developed for low-stakes errors should not have been extended silently to categories where the cost is different. And that the organizations being described bear more of the responsibility than they currently realize, because they control the source material and they are the only party in the chain who can tell whether it is true.
Most of them have never looked.
AUTHOR BIO
Neycho Tepavicharov is co-founder of Flashcloud, a web hosting company. He has spent nearly two decades in the industry, including co-founding and selling a previous hosting business, and writes about infrastructure, AI visibility and the practical realities of running customer-facing systems.
AI gets things wrong. Some categories can’t afford it.
Posted in Commentary with tags Flashcloud on September 17, 2026 by itnerdBy Neycho Tepavicharov, Co-founder, Flashcloud
We have largely accepted that AI assistants get things wrong. Such tolerance is reasonable enough when the stakes are low. A slightly stale restaurant opening time costs someone a wasted trip. A wrong product spec gets corrected at checkout. The error is annoying and self-limiting, but that’s pretty much where it ends.
To be specific about our scope here, this is not about medical advice. That is a separate problem for others more qualified to parse.
This is about the mundane operational facts of a practice – where it is, what it treats, which providers work there, whether it is taking new patients, how you make an appointment. Ordinary directory information, of the kind every business has.
Those facts get read, summarized and repeated by AI assistants the same way restaurant hours do. The difference is what happens when they are wrong.
The Stakes Are Not Evenly Distributed
Someone looking for a healthcare specialist, for example, is usually researching under time pressure, often in an unfamiliar area, and frequently at a moment when they are least equipped to verify what they are told. They ask for a clinic that treats a particular condition and accepts new patients, and they get back two or three names with a line about each.
If the specialty listed is one the practice stopped offering two years ago, the patient calls and, ultimately, gets turned away. If the location is a site the practice closed, they travel to it. If the named provider left, they ask for someone who is not there. None of these are catastrophic in isolation. All of them are worse than the equivalent error about a restaurant, and they happen to people who are already having a bad week.
There is a second-order effect worth noting too. Wrong information about a medical practice tends to be repeated with the same confident tone as correct information, and the person receiving it has no obvious way to assess which they have. The usual signal a human reader would use – a website that visibly has not been touched in years – is stripped out entirely by the time it reaches a summary.
Nobody Involved Can See It Happening
This is the part that makes it hard to fix rather than merely an unfortunate reality.
Practices have almost always learned of wrong information through complaints. Someone turns up at the closed office and rings to say so. But that loop requires the affected person to make contact.
An AI-mediated error can break that loop because the decision may happen before the patient reaches the practice. If the AI assistant leaves the practice out, describes it poorly, points to another provider with clearer information or relies on an outdated listing, the patient may simply move on. The practice never gets a visit, call or form submission, so there is nothing to register. The missed opportunity disappears before it becomes data.
I run a hosting company, so I see a lot of small business websites from the server side, including plenty of medical and dental practices. The pattern is consistent and it’s not due to being careless. A practice website tends to sit between the office manager, whoever built it years ago, and nobody. It gets updated when something forces the issue. Meanwhile it often falls under the radar and becomes the primary source AI uses to describe the practice to prospective patients.
Some practices do not have a working site at all, which is its own version of the same problem – the assistant assembles a description from whatever third-party listings it can find, and the practice has no input into any of it.
Why Healthcare Exposes The General Failure
Healthcare matters here because it makes the larger problem easier to see. AI does not have to invent a wrong answer to cause harm. It can simply repeat outdated information from a source the organization forgot to maintain.
The failure is not that models hallucinate, though they do. It is that they faithfully report stale source material, with no mechanism for signalling age of the information, or confidence that it’s correct, to people who have no way to check. The model is working as designed. The input was wrong, and nothing in the pipeline flags that.
That applies everywhere. Healthcare just makes it legible, because the consequences are concrete enough to notice and the information involved is unambiguously factual. There is no interpretation involved in whether a clinic has moved.
It also suggests where the fix has to sit. Improving the model does not help if the source is wrong. The correction has to happen at the source, which means the organisations being described have to know they are being described, and currently most of them do not.
What That Means Practically
For anyone running or advising a business in a category where accuracy matters, the immediate step is as simple as finding out what is being said.
This takes about twenty minutes. Ask different AI assistants the questions a customer would ask, in the words a customer would use, and note whether you appear and whether the description is right. We wrote up how to do that properly, including which sources to look at once you have the answers, because the sources cited are usually more useful than the answers themselves.
Then assign it to someone. Most of these problems persist not because they are hard but because no individual is responsible for whether the public description of the organization is accurate. That responsibility tends to live as a vague assumption spread across marketing, operations and whoever built the site.
And check that the site actually responds. Systems gathering information move fast across many sources. A site that is slow or briefly unreachable when visited is skipped silently, and whatever less accurate source responded gets used instead. A practice can maintain a scrupulously current website and still be misrepresented because it was unavailable at the wrong moment.
A Reasonable Expectation
None of this is an argument against AI assistants in high-stakes categories. They are useful, people are already using them, and that is not reversing.
It is an argument that the tolerance we developed for low-stakes errors should not have been extended silently to categories where the cost is different. And that the organizations being described bear more of the responsibility than they currently realize, because they control the source material and they are the only party in the chain who can tell whether it is true.
Most of them have never looked.
AUTHOR BIO
Neycho Tepavicharov is co-founder of Flashcloud, a web hosting company. He has spent nearly two decades in the industry, including co-founding and selling a previous hosting business, and writes about infrastructure, AI visibility and the practical realities of running customer-facing systems.
Leave a comment »