Post-hoc Power Analyses

The power (1-β) is computed as a function of

• α,
• the population effect size, and
• the sample size(s) used in a study.

Post-hoc power analyses often make sense after a study has already been conducted. For instance, it becomes possible to assess whether a published statistical test in fact had a fair chance to reject an incorrect H₀.

Importantly, post-hoc analyses, like a priori analyses, require an H₁ effect size specification for the underlying population. They must not be confused with so-called retrospective power analyses in which the effect size is estimated from sample data and used to calculate the "observed power", a sample estimate of the true power. The "observed power" is reported in many frequently used computer programs (e.g., the MANOVA procedure of SPSS).

Retrospective power analyses are based on the highly questionable assumption that the sample effect size is essentially identical to the effect size in the population from which it was drawn. Obviously, this assumption is likely to be false, the more so the smaller the sample. In addition, sample effect sizes are typically biased estimators of their population counterparts. For these reasons, we agree with other critics of retrospective power analyses.

Rather than using retrospective power analyses, researchers should specify population effect sizes on a priori grounds. Effect size specification simply means to define the minimum degree of violation of H₀ a researcher would like to detect with a probability not less than (1-β).

Cohen's (1988) definitions of "small", "medium", and "large" effects can be helpful in such effect size specifications. Wherever available, G*Power provides these conventions as "Tool tips" that may be optained by moving the cursor over the "effect size" input parameter field (see below). However, researchers should be aware of the fact that these conventions may have different meanings for different tests.

References

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.