# AI vs non-AI activation benchmark

> Measure AI feature usage against the core product path and by customer segment so the team sees whether the AI is helping the right users or merely attracting the wrong curiosity.

- Canonical HTML: https://growth.iangoh.com/growth-ideas/ai-vs-non-ai-activation-benchmark/
- Source: [newsletter.posthog.com](https://newsletter.posthog.com/p/what-weve-learned-about-building)
- GrowthDex source hub: [PostHog Newsletter](/sources/posthog-newsletter-newsletter-posthog-com-2/)
- Last checked: 2026-05-26
- Rarity: rare
- Budget: low
- Channels: Analytics, Activation, Positioning
- Stages: ai products, analytics, segmentation, positioning


## Why this can grow

AI usage can look encouraging while still hiding a product mistake. A team may see a lot of interaction and assume the feature is working, when the real pattern is that non-ICP users are playing with it more than the buyers the product actually needs. Benchmarking AI usage against the normal activation path and across segments forces a harder question: is the AI improving the product's real job or only creating extra attention around itself? That view helps teams tune positioning, onboarding, and roadmap bets before they build too much around vanity engagement.

## Ian's take

From scaling consumer platforms across MENA and Southeast Asia, my default is to distrust growth work that only looks good in a slide. My bias is to treat this as a small market test first. Make the audience narrow, make the promise concrete, and let the first real response decide whether it deserves more work. I would run it small enough to learn quickly, then only scale the parts that real users repeat, save, reply to, or buy from. For this tactic, I would watch one clear growth signal before putting more time or budget behind it.

## Action plan

1. Define one narrow startup segment where ai vs non-ai activation benchmark can create a measurable lift.
2. Turn the tactic into one offer, page, campaign, or workflow for the Analytics and Activation channel.
3. Use the evidence from newsletter.posthog.com to set the first version of the message, format, and audience.
4. Launch a small test for 7 to 14 days with one success metric: one measurable growth signal.
5. Review the result, keep the winning message, remove weak variants, and turn the learning into a repeatable growth playbook.

## Source-backed example

PostHog found marketers and product managers used Max AI more often than product engineers, which pushed the team to rethink who the assistant was actually serving.

## Adjacent tactics in the same lane

- [Workflow-first AI demand validation](/growth-ideas/workflow-first-ai-demand-validation/) - same source, 1 shared channel, 1 shared stage
- [AI install wizard for 90-second setup](/growth-ideas/ai-install-wizard-for-90-second-setup/) - same source, 1 shared channel, 1 shared stage
- [Suggested prompts in the empty AI state](/growth-ideas/suggested-prompts-in-empty-ai-state/) - same source, 1 shared channel, 1 shared stage
- [Task-based model routing for AI speed](/growth-ideas/task-based-model-routing-for-ai-speed/) - same source, 1 shared channel, 1 shared stage

## Read GrowthDex essays

Browse the plain-English essay index at [GrowthDex Blog](/blog/).

## Related GrowthDex essays

- [The AI feature only feels smart after the first useful minute](/blog/the-ai-feature-only-feels-smart-after-the-first-useful-minute/) - ai products, activation, product UX

## Advisory

If you want help turning this into a working growth system, Ian Goh offers advisory at https://iangoh.com/advisory.