A Priori Power Analysis

A priori power analyses are done before you conduct an experiment. You have: alpha, the desired power (1-beta), and the effect size of the effect you want to detect. You want to know how many subjects you need to run: the total sample size. For instance, if you want to compare the effects of two treatments administered to two different groups of subjects, you choose A priori as type of power analysis, and t-Test (means) as type of test with the "two-tailed" option selected. Suppose you expect a "large" effect according to Cohen's effect size conventions between the two groups (d = .80), and you want to have alpha = beta = .05 (i.e., power = .95), you punch in these values and click the "Calculate" button to find out that you need N = 84 subjects. If you think this is too much, you might want to have G*Power draw a graph for you to see how the sample size changes as a function of the power of your test, or as a function of the effect size you expect. Simply click on the Draw Graph button. Note that there is a list of tests for fast access to test-specific information.

Post-Hoc Power Analysis

Post-hoc power analyses are done after you or someone else conducted an experiment. You have: alpha, N (the total sample size), and the effect size. You want to know the power of a test to detect this effect. For instance, you tried to replicate a finding that involves a difference between two treatments administered to two different groups of subjects, but failed to find the effect with your sample of 36 subjects (14 in Group 1, and 22 in Group 2). Choose Post-hoc as type of power analysis, and t-Test on means as type of test. Suppose you expect a "medium" effect according to Cohen's effect size conventions between the two groups (delta = .50), and you want to have alpha =.05 for a two-tailed test, you punch in these values (and 14 for n 1, plus 22 for n 2) and click the "Calculate" button to find out that your test's power to detect the specified effect is ridiculously low: 1-beta = .2954. However, you might want to draw a graph using the Draw graph option to see how the power changes as a function of the effect size you expect, or as a function of the alpha-level you want to risk. Note that there is a list of tests for fast access to test-specific information.

Compromise Power Analysis

Compromise power analyses represent a novel concept, and only G*Power provides convenient ways to compute them. Thus, if you ever asked yourself "Why G*Power?", this is one possible answer (accuracy of the algorithms and second-to-none flexibility being other candidates for an answer to this question). You may want to use compromise power analyses primarily in the following two situations: For reasons that are beyond your control (e.g., you are working with clinical populations), your N is too small to satisfy conventional levels of alpha and beta (1-power) given your effect size. Given conventional levels of significance, your N is too large (e.g., you are fitting a model to data aggregated over subjects and items) such that even negligible effects would force you to reject H0. In compromise power analyses, users specify H0, H1 (i.e., the size of the effect to be detected), the test statistic to be used, the maximum possible total sample size, and the ratio q := beta/alpha which specifies the relative seriousness of both errors (cf. Cohen, 1965, 1988, p. 5). The problem is to calculate an optimum critical value for the test statistic which satisfies beta/alpha = q. This optimum critical value can be regarded as a rational compromise between the demands for a low alpha-risk and a large power level, given a fixed sample size. Given appropriate subroutines for computing the noncentral distributions of the relevant test statistics (i.e., the exact distributions of the test statistics if H1 is true, cf. Johnson & Kotz, 1970, chap. 28, 30, and 31), it is relatively easy to implement compromise power analyses using an efficient iterative interval dissection algorithm (cf. Press, Flannery, Teukolsky, & Vetterling, 1988, chap. 9). The question is, therefore, why compromise analyses are missing in the currently available power analysis software. The only reason we can think of is that non-standard results may occur, that is, results that are inconsistent with established conventions of statistical inference. Given some fixed sample size, a compromise power analysis could suggest to choose a critical value which corresponds to, say, alpha = beta = .168. These error probabilities are indeed non-standard, but they may nevertheless be reasonable given the constraints of the research. To illustrate, consider the special case of some substantive hypothesis which implies H0, for instance, the hypothesis of no interaction. Does it make more sense to choose alpha = beta = .168 rather than to insist on the standard level alpha = .05 associated with beta = .623? Obviously, the standard .05 alpha-level makes no sense in this situation, because it implies a risk of almost two-thirds to accept falsely the hypothesis of interest. Therefore, not only a priori and post-hoc analyses, but also compromise power analyses should be offered routinely by software which is designed to serve as a researcher's tool. Note that there is a list of tests for fast access to test-specific information.

Please report suggestions for improvements to Axel Buchner, Franz Faul, or Edgar Erdfelder.