Weekly traces hour for agent quality
Review real AI traces in a standing weekly session and turn the sharpest failures and good catches into eval cases.
Why this can grow a startup
Agent failures are often too specific and situational to notice through synthetic tests alone. A recurring trace review forces the team to watch what users actually asked, where the model drifted, and which interventions felt helpful. Converting those observations into eval cases compounds the learning instead of letting each debugging session disappear into chat history.
Source-backed example
PostHog says the team runs a weekly traces hour, manually reviews real sessions with ratings, and then turns both bad failures and strong interventions into evals so future model or prompt changes do not regress those behaviors.
Source-reported evidence
Source: PostHog Newsletter · Browse PostHog Newsletter tactics
PostHog reviews real rated agent sessions weekly and uses those findings to create future eval cases.
Attributed to the cited source; GrowthDex has not independently verified it. Use this as context, not as a forecast, benchmark, or recommended target.
Evidence scope: claims and metrics are attributed to the linked source; GrowthDex has not independently replicated the reported result. Check the original context and validate fit before applying it. Read the evidence methodology.
Source discovered: May 26, 2026
When to use it
Use this when Product, Retention, Support is relevant to ai products, retention, quality and you can run a bounded test with a low budget.
When not to use it
Do not use it as a substitute for customer evidence, a clear owner, or a measurable stop condition. Local platform rules and market behavior still need checking.
How to do it
- Choose one target segment and define the exact next action this tactic should prompt.
- Adapt the tactic to Product and Retention, using evidence from PostHog Newsletter as context—not as a promised result.
- Record a baseline for: Share of the target cohort repeating the core action within the chosen period. Set a guardrail that would make you pause the test.
- Run a narrow pilot and choose a test window that fits the normal decision or use cycle; do not scale before you have observed the full cycle.
- Compare the signal with its baseline. Scale only if the intended action improves without crossing the guardrail; otherwise revise one variable or stop and record what you learned.
Compare a stack for your situation
Compare source-backed tactics against your product, stage, budget and channel strengths, then choose one experiment to test.
Explore the context
Did you launch this experiment?
Report a launch to help us understand which growth ideas founders put into practice.
Apply this with an operator
Connect activation, customer value, retention, and referral into one measurable loop.
SOURCE HUB
More tactics from PostHog Newsletter
Explore related GrowthDex examples and evidence connected to PostHog Newsletter.
Open PostHog Newsletter source hub