# AI response rating with follow-up context

> Ask users to rate AI responses and collect a short follow-up when they rate a result poorly so tuning work starts from real failures, not guesswork.

- Canonical HTML: https://growth.iangoh.com/growth-ideas/ai-response-rating-with-follow-up-context/
- Source: [newsletter.posthog.com](https://newsletter.posthog.com/p/what-weve-learned-about-building)
- GrowthDex source hub: [PostHog Newsletter](/sources/posthog-newsletter-newsletter-posthog-com-2/)
- Last checked: 2026-05-26
- Rarity: uncommon
- Budget: free
- Channels: Feedback, Retention, Product
- Stages: ai products, retention, feedback, quality


## Why this can grow

AI teams usually know they need feedback, but a thumbs-up or thumbs-down on its own is too thin to drive product changes. The useful moment is right after a weak answer, when the user still remembers what they wanted, what was missing, and how much context the AI ignored. A quick follow-up field turns irritation into training material. That helps the team tighten prompts, routing, context injection, and UI cues around failures that actually cost trust.

## Ian's take

From scaling consumer platforms across MENA and Southeast Asia, my default is to distrust growth work that only looks good in a slide. My bias is to treat this as a small market test first. Make the audience narrow, make the promise concrete, and let the first real response decide whether it deserves more work. For retention, I would watch the second and third use, not just the first click. A tactic is real when it changes a habit. For this tactic, I would watch one clear growth signal before putting more time or budget behind it.

## Action plan

1. Define one narrow startup segment where ai response rating with follow-up context can create a measurable lift.
2. Turn the tactic into one offer, page, campaign, or workflow for the Feedback and Retention channel.
3. Use the evidence from newsletter.posthog.com to set the first version of the message, format, and audience.
4. Launch a small test for 7 to 14 days with one success metric: one measurable growth signal.
5. Review the result, keep the winning message, remove weak variants, and turn the learning into a repeatable growth playbook.

## Source-backed example

PostHog asks users to rate Max AI outputs and requests extra detail on poor ratings so the team can understand the exact shape of low-quality answers.

## Adjacent tactics in the same lane

- [Weekly traces hour for agent quality](/growth-ideas/weekly-traces-hour-for-agent-quality/) - same source, 2 shared channels, 3 shared stages
- [Uncertainty, source, and progress cues in AI UI](/growth-ideas/uncertainty-source-and-progress-cues-in-ai-ui/) - same source, 2 shared channels, 2 shared stages
- [Workflow-first AI demand validation](/growth-ideas/workflow-first-ai-demand-validation/) - same source, 2 shared channels, 1 shared stage
- [Task-based model routing for AI speed](/growth-ideas/task-based-model-routing-for-ai-speed/) - same source, 2 shared channels, 1 shared stage

## Read GrowthDex essays

Browse the plain-English essay index at [GrowthDex Blog](/blog/).

## Related GrowthDex essays

- [The AI feature only feels smart after the first useful minute](/blog/the-ai-feature-only-feels-smart-after-the-first-useful-minute/) - ai products, activation, product UX

## Advisory

If you want help turning this into a working growth system, Ian Goh offers advisory at https://iangoh.com/advisory.