Linear multiple regression: Random model

In multiple regression analyses, we are interested in the relation of a dependent variable Y to m independent factors X = (X₁, ..., X_m). The present procedure refers to the so-called unconditional or random factors model of multiple regression (Gatsonis & Sampson, 1989 Sampson, 1974), that is, it is assumed that Y and X₁ , . . . , X_m are random variables, where (Y, X₁, . . . , X_m) have a joint multivariate normal distribution with a positive definite covariance matrix: 

and mean (µ_Y , µ_X). Without loss of generality we may assume centered variables with µ_Y = 0, µ_Xi = 0. The squared population multiple correlation coefficient between Y and X is given by: 

ρ²_YX = Σ′Y_X Σ⁻¹_X Σ_YX /σ²_Y. 

The regression coefficient vector γ, the analog of β in the fixed model, is given by γ = Σ⁻¹_X Σ_YX. The maximum likelihood estimates of the regression coefficient vector and the residuals are the same under both the fixed and the random model (see Theorem 1 in Sampson, 1974); the models differ, however, with respect to power. The present procedure allows power analyses for the test that the population squared correlations coefficient ρ²_YX has the value ρ²₀ . The null and alternative hypotheses are: 

H₀ : ρ²_YX = ρ²₀ 

H₁ : ρ²_YX X ≠ ρ²₀. 

An important special case is ρ₀ = 0 (corresponding to the assumption that Σ_YX = 0). A commonly used test statistic for this case is F = [(N − m − 1)/p]R²_YX /(1 − R²_YX), which has a central F distribution with df₁ = m and df₂ = N − m − 1. This is the same test statistic as that used in the fixed model. The power differs, however.

Effect size index

The effect size is the population squared correlation coefficient H1 ρ² under the alternative hypothesis. To fully specify the effect size, you also need to specify the population squared correlation coefficient H0 ρ² under the null hypothesis. 

Pressing the Determine button on the left of the effect size label in the main window opens the effect size drawer that may be used to calculate ρ2 either from the confidence interval for the population ρ²_YX given an observed squared multiple correlation R², or from predictor correlations.

Effect size from C.I.

Above you can see an example of how H1 ρ can be determined from the confidence interval computed for an observed R². You have to specify the total sample size, the number of predictors, the observed R² and the confidence level (1 - α) of the confidence interval. In the remaining input field, a relative position inside the confidence interval can be specified that determines the H1 ρ² value. This value can range from 0 to 1, where 0, 0.5 and 1 correspond to the left, central and right positions inside the interval, respectively.

The C.I. lower ρ² and C.I. upper ρ² output fields contain the left and right border of the two-sided 100(1 − α) percent confidence interval for ρ². The Statistical lower bound and Statistical upper bound output fields show the one-sided (0, R) and (L, 1) intervals, respectively. 

Effect size from predictor correlations

By choosing the From predictor correlation matrix option one may compute ρ² from the matrix of correlations among the predictor variables and the correlations between predictors and the dependent variable Y. Pressing the Insert/edit matrix button opens a window in which one can specify 

1. the row vector u containing the correlations between each of the m predictors X_i and the dependent (or outcome) variable Y and 
2. the m × m matrix B of correlations among the predictors (see below). 

f² = ρ²/(1 − ρ2)

ρ² = f^2/(1 + f²)

small f² = 0.02 
medium f² = 0.15 
large f² = 0.35 

small ρ² = 0.02 
medium ρ² = 0.13 
large ρ² = 0.26 

Options

You can switch between an exact procedure for the calculation of the distribution of the squared multiple correlation coefficient ρ² and a three-moment F approximation suggested by Lee (1971, p.123). The latter is slightly faster and may be used to check the results of the exact routine.

Examples

Power and sample size: Example 1

We replicate an example given for the procedure for the fixed model, but now under the assumption that the predictors are not fixed but random samples: We assume that a dependent variable Y is predicted by as Set B of 5 predictors and that ρ²_YX is 0.10, that is, that the 5 predictors account for 10% of the variance of Y. The sample size is N = 95 subjects. What is the power of the F test that ρ²_YX = 0 at α = 0.05? 

We choose the following settings in G*Power:

Select

Type of power analysis: Post hoc

Input parameters

Tail(s): One
H1 ρ²: 0.1
H0 ρ²: 0.0
α err prob: 0.05 
Total sample size: 95 
Number of predictors: 5

Output

Lower critical R²: 0.115170
Upper critical R²: 0.115170
Power (1- β): 0.662627

The output shows that the power of this test is about 0.663 which is slightly lower than the power of 0.674 found in the fixed model. This observation holds in general: The power in the random model is never larger than that found for the same scenario in the fixed model.

Power and sample size: Example 2

We now replicate the test of the hypotheses H₀ : ρ² ≤ 0.3 versus H₁ : ρ² > 0.3 given in Shieh and Kung (2007, p.733), for N = 100, α = 0.05, and m = 5 predictors. We assume that H1 ρ² = 0.4 . The settings and output in this case are:

Select

Type of power analysis: Post hoc

Input parameters

Tail(s): One
H1 ρ²: 0.4
H0 ρ²: 0.3
α err prob: 0.05 
Total sample size: 100
Number of predictors: 5

Output

Lower critical R² : 0.456625
Upper critical R² : 0.456625
Power (1- β): 0.346482

The results show that H₀ should be rejected if the observed R² is larger than 0.457. The power of the test is about 0.346. Assume that we observed R² = 0.5. To calculate the associated p-value we may use the G*power calculator. The syntax for obtaining the CDF of the squared sample multiple correlation coefficient is mr2cdf(R2,ρ2,m+1,N). Thus for the present case we insert 1-mr2cdf(0.5,0.3,6,100) in the calculator. Hitting Calculate gives 0.01278. These values replicate those given in Shieh and Kung (2007).

Power and sample size: Example 3

We now ask for the minimum sample size required for testing the hypothesis H₀: ρ² ≥ 0.2 vs. the specific alternative hypothesis H₁: ρ² = 0.05 with 5 predictors to achieve a power of β = 0.9 and α = 0.05 (Example 2 in Shieh & Kung, 2007). The inputs and outputs are:

Select

Type of power analysis: A priori

Input parameters

Tail(s): One
H1 ρ²: 0.05
H0 ρ² : 0.2
α err prob: 0.05 
Power (1- β ): 0.9
Number of predictors: 5

Output

Lower critical R² : 0.132309
Upper critical R² : 0.132309
Total sample size: 153
Actual power: 0.901051

The results show that N should not be less than 153. This confirms the results in Shieh and Kung (2007).

Using confidence intervals to determine the effect size

Let us assume that in a regression analysis with 5 predictors and N = 50 we observed a squared multiple correlation coefficient R² = 0.3. Let us assume further that we want to use the lower boundary of the 95% confidence interval for ρ² as H1 ρ².

Pressing the Determine button next to the effect size field in the main window opens the effect size drawer. After selecting From confidence interval we specify

Total sample size: 50
Number of predictors: 5
Observed R²: 0.3
Confidence level: 0.95

Finally, we set Rel C.I. pos to use (0=left, 1=right) to 0 to select the left interval border. Pressing Calculate computes the 95% two-sided confidence intervals [0.0337, 0.4606] as well as the lower and upper bounds [0, 4245], [0.0589, 1]. The left boundary of the two-sided interval (0.0337) is transfered to the H1 ρ² field.

Using predictor correlations to determine the effect size

We may use assumptions about the (m × m) correlation matrix between a set of m predictors, and the m correlations between predictor variables and the dependent variable Y to determine ρ². Pressing the Determine button next to the effect size field in the main window opens the effect size drawer.

After selecting From predictor correlations, we insert the number of predictors in the corresponding field and press Insert/edit matrix. This opens a input dialog (see above).

Suppose that we have 4 predictors and that the 4 correlations between X_i and Y are u = (0.3, 0.1, −0.2, 0.2). We select Corr between predictors and outcome and then insert these values.

Assume further that the correlations between X₁ and X₃ and between X₂ and X₄ are 0.5 and 0.2, respectively, whereas all other predictor pairs are uncorrelated. We select Corr between predictors and insert the correlation matrix


Pressing the Calc ρ² button computes ρ² = u B − 1 u′ = 0.297083, which also confirms that B is positive-definite and thus a correct correlation matrix.

Related tests

Linear Multiple Regression: Fixed model, deviation of R² from zero
Linear Multiple Regression: Fixed model, increase of R²

Implementation notes

The procedure uses the exact sampling distribution of the squared multiple correlation coefficient (MRC distribution; Lee, 1971, 1972). The parameters of this distribution are the population squared multiple correlation coefficient ρ^₂, the number of predictors m, and the sample size N. The only difference between the H₀ and H₁ distribution is that the population multiple correlation coefficient is set to H0 ρ² in the former and to H1 ρ² in the latter case.

Several algorithms for the computation of the exact or approximate CDF of the sampling distribution have been proposed (Benton & Krishnamoorthy, 2003; Ding, 1996; Ding & Bargmann, 1991; Lee, 1971, 1972). Benton and Krishnamoorthy (2003) have shown, that the implementation proposed by Ding and Bargmann (1991) (that is used in Dunlap, Xin, & Myers, 2004) may produce grossly false results in some cases. The implementation of Ding (1996) has the disadvantage that it overflows for large sample sizes, because factorials occuring in ratios are explicitly evaluated. This can easily be avoided by using the log of the gamma function in the computation instead.

In G*Power we use the procedure of Benton and Krishnamoorthy (2003) to compute the exact CDF and a modified version of the procedure given in Ding (1996) to compute the exact PDF of the distribution. Optionally, one can choose to use the 3-moment noncentral F approximation proposed by Lee (1971) to compute the CDF. The latter procedure has also been used by Steiger and Fouladi (1992) in their R2 program, which provides similar functionality.

Validation

The power and sample size results were checked against the values produced by R2 (Steiger & Fouladi, 1992), the tables in Gatsonis and Sampson (1989), and results reported in Dunlap et al. (2004) and in Shieh and Kung (2007). Slight deviations from the values computed with R2 were found. These deviations are due to the approximation used in R2, whereas complete correspondence was found in all other tests. The confidence intervals were checked against values computed in R2, the results reported in Shieh and Kung (2007), and the tables given in Mendoza and Stafford (2001).

References

Benton, D., & Krishnamoorthy, K. (2003). Computing discrete mixtures of continuous distributions: noncentral chisquare, noncentral t and the distribution of the square of the sample multiple correlation coefficient. Computational Statistics & Data Analysis, 43, 249-267.

Ding, C. G. (1996). On the computation of the distribution of the square of the sample multiple correlation coefficient. Computational statistics & data analysis, 22, 345-350.

Ding, C. G., & Bargmann, R. E. (1991). Algorithm as 260: Evaluation of the distribution of the square of the sample multiple correlation coefficient. Applied Statistics, 40, 195-198.

Dunlap, W. P., Xin, X., & Myers, L. (2004). Computing aspects of power for multiple regression. Behavior Research Methods, Instruments & Computers, 36, 695-701.

Gatsonis, C., & Sampson, A. R. (1989). Multiple correlation: Exact power and sample size calculations. Psychological Bulletin, 106, 516-524.

Lee, Y. (1971). Some results on the sampling distribution of the multiple correlation coefficient. Journal of the Royal Statistical Society. Series B (Methodological), 33, 117- 130.

Lee, Y. (1972). Tables of the upper percentage points of the multiple correlation coefficient. Biometrika, 59, 179-189.

Mendoza, J., & Stafford, K. (2001). Confidence interval, power calculation, and sample size estimation for the squared multiple correlation coefficient under the fixed and random regression models: A computer program and useful standard tables. Educational & Psychological Measurement, 61, 650-667.

Sampson, A. R. (1974). A tale of two regressions. American Statistical Association, 69, 682-689.

Shieh, G., & Kung, C.-F. (2007). Methodological and computational considerations for multiple correlation analysis. Behavior Research Methods, 39, 731-734.

Steiger, J. H., & Fouladi, R. T. (1992). R2: A computer program for interval estimation, power calculations, sample size estimation, and hypothesis testing in multiple regression. Behavior Research Methods, Instruments, & Computers, 24, 581-582.