PostHog synthetic prompt reset when real user language disagrees
Throw away the prompt set and start over when synthetic tracking stops matching how real customers actually describe the product.
Why this can grow a startup
New channels tempt teams to defend bad history because the data feels scarce and therefore sacred. PostHog took the harder but better route. Once real prompt submissions showed that the tracked prompt set had little overlap with what real users were typing, the team scrapped the old system and rebuilt it. That hurts in the short term, especially when months of trend lines disappear. But keeping a false baseline is worse. It teaches the wrong topics, the wrong comparisons, and the wrong positioning. In fast-changing answer engines, adaptability is not cleanup work. It is the work.
Source-backed example
PostHog says it threw away roughly six months of AI-generated prompt history after realizing those prompts barely overlapped with the real user prompts collected through onboarding.
Source-reported evidence
Source: PostHog: LLMs are picking winners. Here's how to become one. · Browse PostHog: LLMs are picking winners. Here's how to become one. tactics
PostHog reset about six months of AEO tracking after the synthetic prompt set diverged from real prompt language collected from converting users.
Attributed to the cited source; GrowthDex has not independently verified it. Use this as context, not as a forecast, benchmark, or recommended target.
Evidence scope: claims and metrics are attributed to the linked source; GrowthDex has not independently replicated the reported result. Check the original context and validate fit before applying it. Read the evidence methodology.
When to use it
Use this when AI visibility, Analytics, Positioning is relevant to prompt reset, data quality, adaptability and you can run a bounded test with a low budget.
When not to use it
Do not use it as a substitute for customer evidence, a clear owner, or a measurable stop condition. Local platform rules and market behavior still need checking.
How to do it
- Choose one target segment and define the exact next action this tactic should prompt.
- Adapt the tactic to AI visibility and Analytics, using evidence from PostHog: LLMs are picking winners. Here's how to become one. as context—not as a promised result.
- Record a baseline for: Target users who complete the tactic’s intended next action. Set a guardrail that would make you pause the test.
- Run a narrow pilot and choose a test window that fits the normal decision or use cycle; do not scale before you have observed the full cycle.
- Compare the signal with its baseline. Scale only if the intended action improves without crossing the guardrail; otherwise revise one variable or stop and record what you learned.
Compare a stack for your situation
Compare source-backed tactics against your product, stage, budget and channel strengths, then choose one experiment to test.
Explore the context
Did you launch this experiment?
Report a launch to help us understand which growth ideas founders put into practice.
Apply this with an operator
Turn isolated search tactics into a crawlable visibility system tied to demand and proof.