METRICS SELECTION SCENARIO

Product Context: You’re launching a new feed ranking algorithm that optimizes for meaningful interactions rather than simple engagement. Current algorithm sorts by recency with basic engagement signals.

Current Metrics:

• DAU: 12 million
• Session time: 18 min/day
• Engagement rate: 8.5%
• 7-day retention: 52%

User Research: 68% report “scrolling mindlessly” without finding content that matters. Users with meaningful interactions return 23% more frequently.

Your Task: Define North Star, primary metrics (0-3 months), secondary/health metrics, leading indicators (week 1), and long-term measures (6+ months).

FEEDBACK ON YOUR METRICS FRAMEWORK

North Star Recommendation

Meaningful Interactions per DAU per Week

• Definition: Comments ≥10 words + shares with commentary + saves
• Baseline: 0.75 per user/week
• Target: 0.86+ (15% increase)

Statistical Requirements

from scipy.stats import norm
import math

# Primary metric sample size
baseline_rate = 0.023
target_rate = baseline_rate * 1.15

z_alpha, z_beta = 1.96, 0.84
pooled_p = (baseline_rate + target_rate) / 2

n = ((z_alpha + z_beta)**2 * 2 * pooled_p * (1-pooled_p)) / (target_rate - baseline_rate)**2
# Required: ~1.2M per variant
# With 6M DAU per variant: <1 day to power, run 14 days for patterns

# For 90-day retention (4pp lift from 40% to 44%)
# Need ~20k users, 104 days total timeline

Segment-Specific Success Criteria

OVERALL SCORE: 8.5/10

Strong On: Systems thinking, metric hierarchy, business alignment
Practice More: Sample size calculations, instrumentation planning, counter-metrics