Understanding Statistical Power
Educational Tool
Statistical power is the probability of detecting a true effect when one exists. Before conducting a study, power analysis helps determine how large a sample you need. After a study, understanding power helps interpret non-significant results. This tool demonstrates power concepts for simple z and t tests on means.
What is Statistical Power?
Power is the probability that your test will correctly reject a false null hypothesis. In other words, if there really is an effect, power tells you how likely you are to find it.
- • β = Type II error rate (false negative)
- • Conventional target: 80% power
- • Important studies often aim for 90%+
Four Factors of Power
- 1. Effect Size (δ): Larger differences between μ₀ and μ₁ are easier to detect → higher power
- 2. Sample Size (n): More observations → more precise estimates → higher power
- 3. Alpha (α): Higher significance level → easier to reject H₀ → higher power (but more false positives)
- 4. Variability (σ): Less noise in data → clearer signal → higher power
These four factors are mathematically linked: fixing any three determines the fourth.
z-test vs t-test
- • Population σ is known
- • Uses standard normal distribution
- • Rare in practice (σ usually unknown)
- • Good approximation for large n
- • σ estimated from sample (s)
- • Uses t-distribution with df
- • More common in practice
- • Accounts for estimation uncertainty
Test Scenarios
Compare sample mean to a known or hypothesized value μ₀. Example: Is the average height of students different from 170cm?
Compare means of two independent groups with equal sample sizes. Example: Do treatment and control groups differ?
Important Limitations
- Educational purposes only: This tool uses simplified formulas and normal approximations. Do not use for clinical trial design, regulatory submissions, or high-stakes research planning.
- Assumptions: Results assume normality (or large samples), independent observations, equal variances for two-sample tests, and known or well-estimated standard deviations.
- Effect size uncertainty: In practice, the expected effect size is often uncertain. Consider sensitivity analyses across a range of plausible effect sizes.
- For real studies: Use dedicated power analysis software (G*Power, PASS, nQuery, Stata, R packages) and consult with a statistician.
Frequently Asked Questions
Related Tools
Z-Score & P-Value Calculator
Convert between z-scores and p-values for hypothesis testing
Normal Distribution Calculator
Calculate probabilities and quantiles under the normal curve
Confidence Interval Calculator
Build confidence intervals for means and proportions
T-Test Calculator
Run one-sample, two-sample, and paired t-tests
CI for Proportions
Compute confidence intervals for proportions using Wald and Wilson methods
Correlation Significance
Test whether a correlation coefficient is statistically significant