Linear multiple regression: Fixed model, R² increase

H₀ : R²_Y·A,B - R²_Y·A = 0
H₁ : R²_Y·A,B - R²_Y·A > 0.

The directional form of H₁ is due to the fact that R²_Y·A,B, that is, the proportion of variance explained by sets A and B combined, cannot be lower than R²_Y·A, the proportion of variance explained by Set A alone. As will be shown in the examples section, the MRC procedure is quite flexible and can be used as a substitute for some other tests.

Effect size index

The general definition of the effect size index f² used in this procedure is: f² = V_s/V_e, where V_s is the proportion of variance explained by a set of predictors, and V_e is the residual or error variance (V_s + V_e= 1). We now consider two special cases.

Special case 1

In the first special case considered here (Case 1 in Cohen, 1988, p. 407ff.) the proportion of variance explained by the additional predictor Set B is given by V_s = R²_Y·A,B - R²_Y·A and the residual variance by V_e = 1 − R²_Y·A,B. Thus:

f² = (R²_Y·A,B - R²_Y·A)/(1 - R²_Y·A,B).

The quantity R²_Y·A,B - R²_Y·A is also called the semipartial multiple correlation, R²_Y·(B·A). A further interesting quantity is the partial multiple correlation coefficient, which is defined as

R²_{Y B·A} := (R²_Y·A,B - R²_Y·A)/(1 - R²_Y·A) = R²_Y·(B·A)/(1 - R²_Y·A)

Using this definition, f² can alternatively be written in terms of the partial R²:

f² = R²_{Y B·A}/(1 - R²_{Y B·A}).

Special case 2

In the second special case considered here (Case 2 in Cohen, 1988, p. 407ff.) the same effect variance V_s is considered, but it is assumed that there is a third set of predictors C that also accounts for parts of the variance of Y and thus reduces the error variance: V_e = 1 − R²_Y·A,B,C. In this case, the effect size is:

f² = (R²_Y·A,B - R²_Y·A)/(1 - R²_Y·A,B,C).

We may again define a partial R² as

R²_a := = R²_{Y B·A}/(1 - (R²_Y·A,B,C - R²_{Y B·A}))

so that we arrive at

f² = R²_a/(1 - R²_a).

Cohen (1988, p. 412) defines the following conventional values for the effect size f²:

small f² = 0.02
medium f² = 0.15
large f² = 0.35

Pressing the Determine button on the left side of the effect size label in the main window opens the effect size drawer that may be used to calculate f² from V_s and V_e or, alternatively, from the partial R².

Options

This test has no options.

Examples

Basic example for Case 1

We make the following assumptions: A dependent variable Y is predicted by two sets of predictors A and B. The 5 predictors in A alone account for 25% of the variation of Y, thus R²_Y·A = 0.25. Adding the 4 predictors in Set B increases the proportion of variance explained to 0.3, thus R²_Y·A,B = 0.3. We want to calculate the power of a test for the increase due to the inclusion of B, given α = 0.01 and a total sample size of N = 90.

First we use the From variances option in the effect size drawer to calculate the effect size. In the Variance explained by special effect input field we insert R²_Y·A,B − R²_Y·A = 0.3 − 0.25 = 0.05, and as Residual variance we insert 1 − R²_Y·A,B = 1 − 0.3 = 0.7. After clicking on Calculate and transfer to main window we see that this corresponds to a partial R² of about 0.0666 and to an effect size f = 0.07142857. We then set the Numerator df input field in the main window to 4 (the number of predictors in Set B), and Number of predictors to the total number of predictors in sets A and B, that is to 4 + 5 = 9.

This leads to the following analysis in G*Power:

Select

Type of power analysis: Post hoc

Input

Effect size f² : 0.0714286
α err prob: 0.01
Total sample size: 90
Numerator df: 4
Number of predictors: 9

Output

Noncentrality parameter λ: 6.428574
Critical F : 3.563110
Denominator df: 80
Power (1- β ): 0.241297

We find that the power of this test is very low: 1- β is about 0.24. This confirms the result estimated by Cohen (1988, p. 434) in his example 9.10, which uses identical values. It should be noted, however, that (Cohen, 1988) uses an approximation to the correct formula for the noncentrality parameter λ, that in general underestimates the true λ and thus also the true power. In this particular case, Cohen estimates λ = 6.1, which is only slightly lower than the correct value λ = 6.429 given in the output above. By using an A priori analysis, we can compute how large the sample size must be in order to achieve a power of 0.80. We find that the required samples size is N = 242.

Basic example for Case 2

Her we make the following assumptions: A dependent variable Y is predicted by three sets of predictors A, B and C, which stand in the following causal relationship A ⇒ B ⇒ C. The 5 predictors in A alone account for 10% of the variation of Y, thus R²_Y·A = 0.10. Including the 3 predictors in Set B increases the proportion of variance explained to 0.16, thus R²_Y·A,B = 0.16. Considering in addition the 4 predictors in set C increases the explained variance further to 0.2, thus R²_Y·A,B,C = 0.2. We want to calculate the power of a test for the increase in variance explained due to the inclusion of predictor Set B in addition to predictor Set A, given α = 0.01 and a total sample size of N = 200. This is a Case 2 scenario because the hypothesis only involves sets A and B, whereas set C should be included in the calculation of the residual variance.

We use the From variances option in the effect size drawer to calculate the effect size. In the Variance explained by special effect input field we insert R²_Y·A,B − R²_Y·A = 0.16 − 0.1 = 0.06. As Residual variance we insert 1 − R²_Y·A,B,C = 1 − 0.2 = 0.8. Clicking on Calculate and transfer to main window shows that this corresponds to a partial R² = 0.06976744 and to an effect of size f = 0.075. We then set the Numerator df input field in the main window to 3 (the number of predictors in Set B; these cause the potential increase in variance explained). We also set the Number of predictors to the total number of predictors in A, B, and C (which all influence the residual variance), that is, to 5 + 3 + 4 = 12.

This leads to the following analysis in G*Power:

Select

Type of power analysis: Post hoc

Input

Effect size f²:  0.075
α err prob: 0.01
Total sample size: 200
Numerator df: 3
Number of predictors: 12

Output

Noncentrality parameter λ: 15.000000
Critical F : 3.888052
Denominator df: 187
Power (1- β ): 0.766990

We find that  the power of this test is about 0.767. In this case the power is slightly larger than the power value of 0.74 estimated by Cohen (1988, p. 439) in his example 9.13, which uses identical values. This is due to the fact that his approximation for λ = 14.3 underestimates the true value λ = 15 given in the output above.

Example showing relations to factorial ANOVA designs

We assume a 2 × 3 × 4 design with three factors U, V, and W. We want to test main effects, two-way interactions (U × V, U × W, V × W) and the three-way interaction (U × V × W). We may use the ANOVA: Fixed effects, special, main effects and interactions procedure in G*Power to do this analysis (see the corresponding entry in the manual for details). As an example, we consider the test of the V × W interaction. Assuming that the Variance explained by the (in this case) interaction = 0.422, and that Error variance = 6.75. This leads to an effect size of f = 0.25 (a medium effect size according to Cohen, 1988). The Numberator df = 6 corresponds to (levels of V - 1)(levels of W - 1). The Number of groups is 24, that is, the total number of cells (2 · 3 · 4 = 24) in the design. With α = 0.05 and a total sample size of N = 120 we compute a power of 0.470.

We now demonstrate how these analyses can be done unsing the MRC procedure. A factor with k levels corresponds to k − 1 predictors in the MRC analysis. Thus, the number of predictors is 1, 2, and 3 for the three factors U, V, and W. The number of predictors in interactions is the product of the number of predictors involved. The V × W interaction, for instance, corresponds to a set of (3 − 1) · (4 − 1) = 2· 3 = 6 predictors.

To test an effect with MRC we need to isolate the relative contribution of this source to the total variance, that is, we need to determine V_s. We illustrate this for the V × W interaction. In this case we must find R²_Y·V×W by excluding from R²_Y·V,W,V×W (the proportion of variance that is explained by V, W, and V × W together) the contribution of the main effects, that is, R²_Y·V×W = R²_Y·V,W,V×W − R²_Y·V,W. The residual variance V_e is the variance of Y from which the variance of all sources in the design have been removed.

This is a Case 2 scenario, in which V × W corresponds to Set B with 2 · 3 = 6 predictors, V ∪ W corresponds to Set A with 2 + 3 = 5 predictors, and all other sources of variance, that is U, U×V, U×W, U×V×W, correspond to set C with (1 +(1 · 2) +(1 · 3) +(1 · 2 · 3)) = 1 + 2 + 3 + 6 = 12 predictors. Thus the total number of predictors is (6 + 5 + 12) = 23. Note that the total number of predictors is always equal to (the number of cells in the design -1).

We now specify these contributions numerically: R²_Y·A,B,C = 0.325, R²_Y·A,B − R²_Y˙A = R²_Y·V×W = 0.0422. Inserting these values in the effect size dialog (Variance explained by special effect = 0.0422, Residual variance = 1 - 0.325 = 0.675) yields an effect size of f² = 0.06251852 (compare these values with those chosen in the ANOVA analysis and note that f² = 0.25² = 0.0625). To calculate the power, we set Numerator df = 6 (that is, equal to the number of predictors in Set B), and Number of predictors = 23 (that is, equal to the total number of predictors). With α = 0.05 and N = 120, we get—as expected—the same power of 0.470 that we also arrive at in the equivalent ANOVA analysis.

Related tests

Linear multiple regression: Fixed model, R² deviation from zero
ANOVA: Fixed effects, special, main effect and interactions

Implementation notes

The H₀ distribution is the central F distribution with numerator degrees of freedom df₁ = q, and denominator degrees of freedom df₂ = N − p − 1, where N is the sample size, q the number of predictors in Set B, which may cause an increase in explained variance, and p is the total number of predictors. In Case 1 (see the "Effect size index" section above), p = q + w. In case 2, p = q + w + v, where w is the number of predictors in Set A, and v is the number of predictors in set C. The H₁ distribution is the noncentral F distribution with the same degrees of freedom and noncentrality parameter λ = f² · N.

Validation

The results were checked against the values produced by GPower 2.0 and those produced by PASS (Hintze, 2006). Slight deviations were found to the values tabulated in Cohen (1988). This is due to an approximation used by Cohen (1988) that underestimates the noncentrality parameter λ and therefore also the power. This issue is discussed more thoroughly in Erdfelder, Faul, and Buchner (1996).

References

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

Erdfelder, E., Faul, F., & Buchner, A. (1996). GPOWER: A general power analysis program. Behavior Research Methods, Instruments, & Computers, 28, 1-11.

Hintze, J. (2006). NCSS, PASS, and GESS. Kaysville, Utah: NCSS.