TP509: The Smart Intern Problem

Somewhere right now, a survey is being answered by someone who does not exist. Sometimes that is fraud. Sometimes it is the product. Synthetic respondents, digital twins and AI personas all start from real human data. None of them is a real person answering your question today. The ideas themselves are old. Personas came out of software design in the late 1990s, and digital twins came out of engineering. What changed is that both are now pointed at people, and the AI answers on their behalf. The early tests lean one way. Synthetic answers drift toward the most common response, and the unusual answers get smoothed out. In one vendor's test of coffee habits, the people who never drink coffee disappeared from the synthetic sample entirely. In healthcare, that missing group is the rural patient, the person with a rare condition and the person who has never used your health system. Those are often the questions that matter most. Follow that drift far enough and it becomes a loop. Cheaper research means less real research gets fielded. That thins the data the models learn from, and it thins the experience junior researchers need to recognize a wrong answer. Human data carries its own risk too. Federal prosecutors charged people tied to two research firms with selling fabricated survey responses for years. Bad data goes in, and clean-looking synthetic data comes out. Chris Boyer and Reed Smith leave one question open: can a new researcher build judgment by grading AI output, or does that instinct only come from reading thousands of real answers? Rob Klein and Eva Gott make the case for a hybrid approach anchored in real respondents. Synthetic panels can stretch a small market or a niche audience and give a quick read on creative. Understanding how a cancer patient feels during treatment still takes a person who can explain why. Rob puts the limit plainly. AI has no peripheral vision. It sees only the data in front of it, which makes it smart and naive at the same time. If your health system stopped paying for real respondents tomorrow, who would notice first? Healthcare's Long Walk Toward the Patient, the free eBook marking 500 episodes, plugged in the open: https://www.touchpointpodcastbook.com Klein & Partners, Rob Klein's national healthcare market research firm: https://www.kleinandpartners.com thinqinsights, Eva Gott's custom market research firm and a strategic partner of Klein & Partners since January 2024: https://www.thinqinsights.com Eva Gott, author profile on Greenbook listing her as Head of Research at thinqinsights: https://www.greenbook.org/insights/authors/eva-gott Klein & Partners and thinqinsights partnership announcement, January 2024: https://www.mrweb.com/drno/news36191.htm HCIC 2026, the Healthcare Interactive Conference, October 25 to 28 2026 at the Omni Orlando ChampionsGate. Rob is presenting on the website journey and AI with a colleague from Virtua Health: https://hcic-prod.enqbator.com/ Argyle and colleagues at Brigham Young University, "Out of One, Many: Using Language Models to Simulate Human Samples," Political Analysis, 2023. The paper that named silicon sampling: https://www.cambridge.org/core/product/035D7C8A55B237942FB6DBAD7CAA4E49 Shumailov and colleagues, "AI models collapse when trained on recursively generated data," Nature, July 2024. The model collapse study cited in Segment 2: https://www.nature.com/articles/s41586-024-07566-y U.S. Department of Justice, District of New Hampshire. Indictment of eight defendants in a scheme to bill about $10 million in fraudulent market survey data through Op4G and Slice, April 15 2025: https://www.justice.gov/usao-nh/pr/eight-defendants-indicted-international-conspiracy-bill-10-million-fraudulent-market U.S. Department of Justice, third guilty plea in the same case, March 5 2026. Source of the figure that fabricated data drove about 90% of the two firms' revenue: https://www.justice.gov/usao-nh/pr/third-man-pleads-guilty-role-10-million-international-fraudulent-market-survey Jang, Lee and Kim at KAIST, "Instruction-Tuned Language Models Cannot Sample fr