F-Test for Analyses of Variance (ANOVA)

We can easily do power analyses for single-factor and multi-factor experiments. In G*Power, you select F-Test (ANOVA), Global for ANOVA, fixed effects: Single-factor designs, or F-Test (ANOVA), Special for ANOVA, fixed effects: Multi-factor designs, ANOVA, fixed effects: Planned comparisons, and Analyses of covariance (ANCOVA). Note: For F-Test (ANOVA) (as well as for F-Test (MCR)), you can choose whether you want to perform power analyses for global (i.e., omnibus) tests or for special tests. Global test is the default option. This test refers to the H0 that all means in the design are equal (ANOVA) or that all regression coefficients (next to the additive constant) are zero (MCR). Random effects ANOVAs and mixed effects ANOVAs are not considered. We may add them at a later time, however. A discussion of how to do power analyses for repeated measures ANOVAs and MANOVAs can be found in the Other F-Tests section. For the ANOVA designs, we will use the effect size index f (Cohen, 1977). The relation of f to the noncentrality parameter lambda is given by lambda = f2 * N.

ANOVA, fixed Effects: Single-Factor Design

H0 is that the population means in k conditions are identical. More formally

H0 :	k sum (mi - m)2 = 0 , i=1
H1:	k sum (mi - m)2 = c, c > 0. i=1

where k is the number of conditions, mi is the mean in condition i, and m is the grand mean. You can compute the effect size index from group means and s which is assumed to be constant across groups by clicking on "Calc 'f'". We will spare you the formula behind that. (Just that much: You can save a lot of time when you use the "Calc 'f'" option.)

Example

We compare 10 groups, and we have reason to expect a "medium" effect size (f = .25). How many subjects do we need?

Select:	Type of Power Analysis:	A priori
	Type of Test:	F-Test (ANOVA), Global
		Accuracy mode calculation

Input:	Alpha:	.05
	Power (1-beta):	.95
	Effect size "f":	.25	(Note that in G*Power you can compute f directly from the population means and the population standard deviation sigma; simply click "Calc f" after selecting "F-Test (ANOVA), Global".)
	Groups:	10	(This is a new item that pops up only when you do power analyses for ANOVAs.)

Result:	Total sample size:	390
	Actual power:	0.9524
	Critical F:	F(9,380) = 1.9045
	Lambda:	24.1237

Thus, we need 39 subjects in each of the 10 groups. What if we had only 200 subjects available? Assuming that both alpha and beta are equally serious (i.e., the ratio q := beta/alpha = 1) which probably is the default in basic research, we can compute the following compromise power analysis:

Select:	Type of Power Analysis:	Compromise
	Type of Test:	F-Test (ANOVA), Global Accuracy
		Accuracy mode calculation

Input:	Total sample size:	200
	Effect size "f":	0.25
	Beta/alpha ratio:	1

Result:	Alpha:	0.1592
	Power (1-beta):	0.8408
	Critical F:	1.4762
	Lambda:	12.50000

ANOVA, Fixed Effects: Multi-Factor Designs

In multi-factor designs, we want to determine separately the power for the main effects and for the interactions involved. For main effects, the H0, the interpretation of the effect size index f, and the procedure are basically the same as for single-factor designs. The major difference is that the numerator df (df = degrees of freedom) are reduced relative to a single-factor design because other factors have to be taken into account. Thus, the only new part is that you need to specify, as the "Groups", all cells of your multi-factor design, and as numerator df (the new item for this type of power analysis) you enter i - 1, where i represents the levels of the specific factor to be tested. Note that there may be considerable differences between the power analysis values as determined by G*Power and those determined according to the "approximations" suggested by Cohen (1977, p. 365). G*Power is correct, while Cohen's approximations systematically underestimate the power. This problem with Cohen's approximation method is described in more detail in the context of our description of the accuracy of the algorithms used in G*Power.

Example 1

Let us assume we have a 3 x 5 design in which Factor A has 3 levels and Factor B has 5 levels. We first want to compute a power analysis for main effect A:

Select:	Type of Power Analysis:	Post-hoc
	Type of Test:	F-Test (ANOVA), Special
		Accuracy mode calculation

Input:	Alpha:	.05
	Effect size "f":	.25
	Total sample size:	270
	Groups:	15	(That is, all cells in your 3x5 design, thus 15 because there are 3 * 5 levels for Factor A and Factor B, respectively.)
	Numerator DF:	2	(Factor A has 3 levels, thus the test of the main effect of Factor A has 3-1=2 df.)

Result:	Power (1-beta):	0.9637 (Yeah!)
	Critical F:	F(2,255) = 3.0312
	Lambda:	16.8750

Example 2

Assume that your H0 states that there is no interaction between A and B. How do you perform a power analysis for this case? The number of Groups is again 3 * 5 = 15. The numerator df is (3-1) * (5-1) = 8. If you enter these values and leave the rest as it was for the main effect, then this is the result:

Result:	Power (1-beta):	0.8396
	Critical F:	F(8,255) = 1.9748
	Lambda:	16.8750

To extend this further, assume that you have a 3 x 4 x 6 design with factors A, B, and C. You test the main and interaction effects of this design using the following values (assuming alpha = .05, effect size f = .25, and a total sample size of 288):

	Effects	Groups:	Numerator df:	Power (1-beta):
	A	72	2	0.9727
	B	72	3	0.9557
	C	72	5	0.9197
	A x B	72	6 (2 * 3)	0.9013
	A x C	72	10 (2 * 5)	0.8290
	B x C	72	15 (3 * 5)	0.7469
	A x B x C	72	30 (2 * 3 * 5)	0.5630

Example 3

So far, we have limited the discussion to post-hoc power analyses. However, in planning a multi-factor design, we want to know how many participants we need to recruit for our experiment. How do we proceed in that case? Essentially, our decisions involve the following steps: Step 1: We compute a priori power analyses for the statistical tests of the effects of all factors and interactions that are interesting from a theoretical point of view. We ignore all other factors and interactions. Step 2: Case 1: One factor or interaction (henceforth our critical factor or interaction) is more important for our research question than all other factors or interactions. Two alternatives are possible: Our critical factor or interaction is the one associated with the largest sample size as determined in Step 1. We use that sample size. As a rule, we will be on the safe side with all other relevant factors and interactions. Our critical factor or interaction is the not the one associated with the largest sample size as determined in Step 1. We need to do some more work: We take the sample size as suggested for the critical factor or interaction and perform post-hoc power analyses for all other factors or interactions that are theoretically relevant. If we can live with the error probabilities associated with the statistical tests of the effects of these factors, then we are done. If we are not happy with the error probabilities, we try to increase the sample size up to the level at which we find both the error probabilities and the resource demands acceptable. Case 2: All factors and interactions are equally important. We use the largest sample size as determined in Step 1. As a rule, we will be on the safe side with all other relevant factors and interactions. (Note that Case 2 and Case 1.1 lead to the same result.) Let us return to our 3 x 5 design in which Factor A has 3 levels and Factor B has 5 levels. For simplicity, we assume that we want to detect effects of size f = .40 for the two main effects and the interaction given alpha = beta = .05. The relevant a priori power analyses suggest the following sample sizes:

	Effects	Groups:	Numerator df:	Total sample size:
	A	15	2	105*
	B	15	4	135*
	A x B	15	8	165*

* Note that the total sample size values produced by G*Power are somewhat smaller. However, we use the next largest number that can be divided by 15 because our design has 15 cells and we wish to assure that the n's in all cells are equal.

In a Case 2 situation, we would need a total sample size of 165. Given that our assumptions about alpha and the effect size remained unchanged, a total sample size of 165 would imply power values > .99 for tests of the effects of Factors A and B. This result is a dream come true! Alternatively, let us assume a Case 1 situation in which Factor B is the most important factor from a theoretical point of view. What would the implications be of accepting 135 as the total sample size? Given that our assumptions about alpha and the effect size remained unchanged, a total sample size of 135 would imply power values of . 9890 and .9195 for tests of the effects of Factors A and the A x B interaction, respectively. This result is certainly acceptable and we may decide to use 135 as the total sample size.

ANOVA, Planned Comparisons

With planned comparisons, the H0 is that the contrasts among the means do not explain, in the dependent variable, any variance which has not already been accounted for by other sources of the effect. The effect size f is defined as _______________ / \ / R2p f = / ______________ / \/ ( 1 - R2p)   where R2p is the partial multiple correlation between the dependent variable and the variable(s) coding the contrast among the means. In G*Power, click "Calc F" after selecting "F-Test (ANOVA), Special" to calculate f from the partial multiple correlation (referred to as partial eta-square in G*Power). For a power analysis, it does not matter whether the contrasts are orthogonal or not. However, note that f does not only depend on the population means but also on the correlations among the contrast variables.

Example

Assume you have a Factor A with 4 levels. We want to determine whether the effect of A on our dependent variable Y is linear, but not quadratic or cubic. You can code A into 4-1=3 orthogonal contrast variables as follows.

	x1, linear	-3	-1	1	3
	x2, quadratic	1	-1	-1	1
	x3, cubic	-1	3	-3	1

Assume that your H1 specifies that R2p = .20 for the linear contrast (x1). Thus, f = .1667. If you have 60 subjects in your experiment (i.e., 15 in each of the four groups), what are the alpha and beta error probabilities if both types of errors are equally important (i.e., the ratio q := beta/alpha = 1)?

Select:	Type of Power Analysis:	Compromise
	Type of Test:	F-Test (ANOVA), Special
		Accuracy mode calculation

Input:	Total sample size:	60
	Effect size "f":	0.2857
	Beta/alpha ratio:	1

Result:	Alpha:	0.1888
	Power (1-beta):	0.8112
	Critical F:	F(1,56) = 1.7700
	Lambda:	4.8975

Analyses of Covariance (ANCOVA)

In an analysis of covariance, we replace a dependent variable Y by a corrected dependent variable Y' which we arrive at by partialling out the linear relation between Y and a set X = (Xa, ... , Xq) of q covariates Xi (Cohen, 1977, p. 379). Y' = Y - bi (Xi-m(Xi)), i = 1 ... q, where bi is the regression weight of Y on Xi (bi is constant across all populations), Xi is the covariate i (i.e., Xi may be different in each of the populations), and m(Xi) is the grand population mean of the concomitant variable or covariate i. In other words, covariate Xi differs in each of the populations we look at, but its relation to Y and, hence, its regression weight bi is the same in all of those populations. The analysis of covariance is essentially an analysis of variance of the Y' measures. However, we need to adjust the denominator df, which is why we need to select the "F-Test (ANOVA), Special" option. If k is the number of cells of your design, choose groups = k + q (q is the number of covariates in your design). In this way, the denominator df are reduced appropriately because G*Power assumes that denominator df = N - groups. If the correlation between Y and the covariates is substantial, then the power of your statistical test is increased. This is so because the within-population standard deviation sigmaY' in the denominator of the F ratio is smaller than sigmaY. Specifically, where r is the (multiple) population correlation between Y' and Y, we find that ________ / \ sigmaY' = sigmaY * \/ 1 - r2 The numerator does not decrease correspondingly. It may even increase. Example Assume a 2 x 3 design. A covariate X has been partialled out of a dependent variable Y'. We want to detect 'large' effects (f = .40) according to Cohen's effect size conventions for Factor B which has 3 levels. We had 60 subjects, and we decide that alpha = .05. What is the power of the F-test in this situation?

Select:	Type of Power Analysis:	Post-hoc
	Type of Test:	F-Test (ANOVA), Special
		Accuracy mode calculation

Input:	Alpha:	.05
	Effect size "f":	.40
	Total sample size:	60
	Groups:	7	(That is, 2 * 3 + 1 = 7.)
	Numerator DF:	2	(Factor B has 3 levels, thus the test of the main effect of Factor B has 3-1=2 df.)

Result:	Power (1-beta):	0.7740
	Critical F:	F(2,53) = 3.1716
	Lambda:	9.6000

Please report suggestions for improvements to Axel Buchner, Franz Faul, or Edgar Erdfelder.