Power Analysis Sample Size Calculator

Pro v1.0.0 1 view

Calculate required sample sizes and statistical power for research studies and A/B tests. Supports t-tests, ANOVA, regression, proportions, and correlations with sensitivity analysis and plain-language explanations.

What You Get

Prevent underpowered studies and wasted data collection by calculating exact sample sizes before research begins, with sensitivity analysis showing tradeoffs.

The Problem

Planning research studies and A/B tests requires complex statistical calculations to determine proper sample sizes. Without power analysis, studies risk being underpowered (wasting months of data collection) or overpowered (wasting resources). Most researchers lack access to statistical software like G*Power or the expertise to use it correctly.

The Solution

This skill automates sample size and power calculations using Python's statsmodels library. It supports five common study designs: proportion tests for A/B testing, t-tests for comparing two groups, ANOVA for multiple groups, regression analysis, and correlation studies. The calculator accepts study parameters, computes required sample sizes or achieved power, generates sensitivity analyses showing how requirements change with different assumptions, and provides plain-language interpretations with actionable recommendations. For A/B tests, it includes critical warnings about early stopping and false positive inflation.

How It Works

  1. 1 Identify study design (proportion, t-test, ANOVA, regression, or correlation) and gather parameters
  2. 2 Run power_calculator.py script with study-specific arguments to compute sample size or power
  3. 3 Review sensitivity analysis showing sample size requirements across different effect sizes and power levels
  4. 4 Interpret results with plain-language explanation of statistical concepts and practical recommendations
  5. 5 For A/B tests, calculate test duration based on daily traffic and emphasize proper stopping rules

What You'll Need

  • Python 3 with statsmodels, numpy, and scipy libraries
  • Study design specification (proportion test, t-test, ANOVA, regression, or correlation)
  • Effect size estimate from literature, pilot data, or domain expertise
  • For A/B tests: baseline conversion rate and minimum detectable relative effect
  • Optional: daily traffic for duration estimates or existing sample size for post-hoc analysis

Get This Skill

Requires Pro subscription ($9/month)

Have Feedback?

Help us improve this skill by sharing your ideas and suggestions.

Request Improvements