Correlations: Two dependent Pearson r's

This procedure provides power analyses for tests of the hypothesis that two dependent Pearson correlation coefficients ρ_a,b and ρ_c,d are identical. The corresponding test statistics Z1⋆ and Z2⋆ were proposed by Dunn and Clark (1969) and are described in Equations (11) and (12) in Steiger (1980) (for more details on these tests, see implementation notes below).

Two correlation coefficients ρ_a,b and ρ_c,d are dependent if at least one of the four possible correlation coefficients ρ_a,c, ρ_a,d, ρ_b,c, and ρ_b,d between other pairs of the four data sets a, b, c, and d is non-zero.

Thus, in the general case where a, b, c, and d are different data sets, we do not only have to consider the two correlations under scrutiny but also four additional correlations.

In the special case in which two of the data sets are identical, the two correlations are obviously always dependent, because at least one of the four additional correlations mentioned above is exactly 1. Two other of the four additional correlations are identical to the two correlations under test. Thus, there remains only one additional correlation that can be freely specified. In this special case we denote the H₀ correlation ρ_a,b , the H₁ correlation ρ_a,c and the additional correlation ρ_b,c.

It is convenient to describe the general case and the special case by two corresponding correlations matrices (that may be sub-matrices of a larger correlations matrix): A 4 × 4-matrix in the general case of four different data sets ('no common index'):




and a 3 × 3 matrix in the special case in which one of the data sets is identical in both correlations ('common index'):



Note: The values for ρ_x,y in matrices C₁ and C₂ cannot be chosen arbitrarily between -1 and 1. This can be seen by considering matrix C₂: We cannot, for instance, choose ρ_a,b = −ρ_a,c and ρ_b,c = 1.0 because the latter choice implies that the other two correlations are identical. It is, however, generally less easy to decide whether a given matrix is a valid correlation matrix. In more complex cases the following formal criterion can be used: A given symmetric matrix is a valid correlation matrix if and only if the matrix is positive semi-definite, that is, if all eigenvalues of the matrix are non-negative.

The null hypothesis in the general case with 'no common index' states that ρ_a,b = ρ_c,d. The (two-sided) alternative hypothesis is that these correlation coefficients are different: ρ_a,b ≠ ρ_c,d:

H₀ : ρ_a,b − ρ_c,d = 0
H₁ : ρ_a,b − ρ_c,d ≠ 0.

Here, G*Power refers to the test Z2⋆ described in Equation (12) in Steiger (1980).

The null hypothesis in the special case of a 'common index' states that ρ_a,b = ρ_a,c. The (two-sided) alternative hypothesis is that these correlation coefficients are different: ρ_a,b = ρ_a,c:

H₀ : ρ_a,b − ρ_a,c = 0
H₁ : ρ_a,b − ρ_a,c ≠ 0.

Here, G*Power refers to the test Z1⋆ described in Equation (12) in Steiger (1980).

If the direction of the deviation ρ_a,b − ρ_c,d (in the 'no common index' case) or ρ_a,b − ρ_a,c (in the 'common index' case) cannot be predicted a priori, a two-sided ('two-tailed') test should be used. Otherwise a one-sided test is adequate.

Effect size index

In this procedure the correlation coefficient assumed under H₁ is used as effect size, that is, ρ_c,d in the general case of 'no common index' and ρ_a,c in the special case of a 'common index'.

To fully specify the effect size, the following additional inputs are required:

• ρ_a,b, the correlation coefficient assumed under H₀, and
• all other relevant correlation coefficients that specify the dependency between the correlations assumed in H₀ and H₁:
  
• ρ_a,c , ρ_a,d , ρ_b,c , ρ_b,d in the general case of 'no common index', and
  
• ρ_b,c in the 'common index' case.

G*Power requires the correlations assumed under H₀ and H₁ to be within the interval [−0.000001, 0.000001]. The additional correlations must be within the interval [−1, 1]. In a priori analyses, zero effect sizes are not allowed, because this would imply an infinite sample size. In this case the additional restriction |ρ_a,b − ρ_c,d| > 0.000001 (or |ρ_a,b − ρ_a,c | > 0.000001) holds.

Why do we not use q, the effect size proposed by Cohen (1988) for the case of two independent correlations? The effect size q is defined as a difference between two 'Fisher z'-transformed correlation coefficients: q = z₁ − z₂, with z₁ = ln((1 + ρ₁)/(1 − ρ₁))/2, z₂ = ln((1 + ρ₂)/(1 − ρ₂))/2. The choice of q as effect size is sensible for tests of independent correlations because in this case the power of the test does not depend on the absolute value of the correlation coefficient assumed under H₀. The power of the test depends only on the difference q between the transformed correlations under H₀ and H₁. This is no longer true for dependent correlations. We therefore used the effect size described above. (See the implementation section for a more thorough discussion of these issues.)

Although the power is not strictly independent of the value of the correlation coefficient under H₀, the deviations are usually relatively small and it may therefore be convenient to use the definition of q to specify the correlation under H₁ for a given correlation under H₀. In this way, one can relate to the effect size conventions for q defined by Cohen (1969, p. 109ff) for independent correlations:

small q = 0.1
medium q = 0.3
large q = 0.5

The effect size drawer, which can be opened by pressing the Determine button on the left side of the effect size label, can be used to do this calculation:

Options

This test has no options.

Examples

We assume the following correlation matrix in the population:

General case: No common index

We want to perform an a priori analysis for a one-sided test of whether ρ_1,4 = ρ_2,3 or whether ρ_1,4 < ρ_2,3 holds.

With respect to the notation used in G*Power we have the following identities a = 1, b = 4, c = 2, and d = 3. Thus wet get:

H₀ correlation:

ρ_a,b = ρ_1,4 = 0.1

H₁ correlation:

ρ_c,d = ρ_2,3 = 0.2,
ρ_a,c = ρ_1,2 = 0.5,
ρ_a,d = ρ_1,3 = 0.4,
ρ_b,c = ρ_4,2 = −0.4, and
ρ_b,d = ρ_4,3 = 0.8.

We want to know how large our samples need to be in order to achieve the error levels α = 0.05 and β = 0.2. We choose the procedure 'Correlations: Two independent Pearson r 's (common index)' and set:

Select

Type of power analysis: A priori

Input

Tail(s): one
H1 corr ρ_cd: 0.2
α err prob: 0.05
Power (1-β err prob): 0.8
H0 Corr ρ_ab: 0.1
Corr ρ_ac: 0.5
Corr ρ_ad: 0.4
Corr ρ_bc: -0.4
Corr ρ_bd: 0.8

Output

Critical z : 1.644854
Sample Size: 886
Actual Power: 0.800093

We find that the sample size in each group needs to be N = 886. How large would N be if we instead assumed that ρ_1,4 and ρ_2,3 were independent, that is, that ρ_ac = ρ_ad = ρ_bc = ρ_bd = 0? To calculate this value, we may set the corresponding input fields to 0, or, alternatively, use the procedure for independent correlations with an Allocation ratio N2/N1 = 1. In either case, we find a considerably larger sample size of N = 1183 per data set (i.e. we correlate data vectors of length N = 1183). This shows that the power of the test increases considerably if we take dependencies between correlations coefficients into account.

If we try to change the correlation ρ_bd from 0.8 to 0.9, G*Power shows an error message stating: 'The correlation matrix is not valid, that is, not positive semi-definite'. This indicates that the matrix C_p with ρ_3,4 changed to 0.9 is not a possible correlation matrix.

Special case: Common index

Assuming again the population correlation matrix C_p shown above, we want to do an a priori analysis for the test of whether ρ_1,3 = ρ_2,3 or whether ρ_1,3 > ρ_2,3 holds. With respect to the notation used in G *Power we have the following identities: a = 3 (the common index), b = 1 (the index of the second data set entering in the correlation assumed under H₀, here ρ_a,b = ρ_3,1), and c = 2 (the index of the remaining data set).

Thus, we get:

H0 correlation:

ρ_a,b = ρ_3,1 = 0.4

H1 correlation

ρ_a,c = ρ_3,2 = 0.2, and
ρ_b,c = ρ_1,2 = 0.5.

For this effect size we want to calculate how large our sample size needs to be in order to achieve  error levels of α = 0.05 and β = 0.2. We choose the procedure 'Correlations: Two independent Pearson r 's (common index)' and set:

Select

Type of power analysis: A priori

Input

Tail(s): one
H1 corr ρ_ac: 0.2
α err prob: 0.05
Power (1-β err prob): 0.8
H0 Corr ρ_ab: 0.4
Corr ρ_bc: 0.5

Output

Critical z : 1.644854
Sample Size: 144
Actual Power: 0.801161

The answer is that we need sample sizes of 144 in each group (i.e. the correlations are calculated between data vectors of length N = 144).

Sensitivity analyses

We now assume a scenario that is identical to that described above, with the exception that ρ_b,c = −0.6. We want to know the minimum H₁ correlation ρ_a,c that we can detect with α = 0.05 and β = 0.2 given a sample size N = 144. In this sensitivity analysis, we have in general two possible solutions, namely one for ρ_a,c ≤ ρ_a,b and one for ρ_a,c ≥ ρ_a,b. The relevant settings for the former case are:

Select

Type of power analysis: Sensitivity

Input

Tail(s): one
Effect direction: ρ_ac ≤ ρ_ab
α err prob: 0.05
Power (1-β err prob): 0.8
Sample Size: 144
H0 Corr ρ_ab: 0.4
Corr ρ_bc: -0.6

Output

Critical z : -1.644854
H1 corr ρ_ac: 0.047702

The result is that the error levels are as requested or lower if the H₁ correlation ρ_a,c is equal to or lower than 0.047702.

We now try to find the corresponding H₁ correlation that is larger than ρ_a,b = 0.4. To this end, we change the effect size direction in the settings shown above, that is, we choose ρ_a,c ≥ ρ_a,b. In this case, however, G*Power shows an error message, indicating that no solution was found. The reason is that there is no H₁ correlation ρ_a,c ≥ ρ_a,b that leads to a valid (i.e. positive semi-definite) correlation matrix and simultaneously ensures the requested error levels. To indicate a missing result, the output for the H₁ correlation is set to the nonsensical value 2.

In both the general case with 'no common index' and in the special 'common index' case G*Power checks whether the correlation matrix is valid and shows a H₁ correlation of 2 if no solution is found. This is also true in the X-Y plot if the H₁ correlation is the dependent variable: Combinations of input values for which no solution can be found show up with a nonsensical value of 2.

Related tests

Correlation: Bivariate normal model
Correlations: Point biserial model
Correlations: Two independent Pearson r 's

Implementation notes

Background

Let X₁ , . . . , X_K denote multinormally distributed random variables with mean vector µ and covariance matrix C. A sample of size N from this K-dimensional distribution leads to a N × K data matrix, and pair-wise correlation of all columns to a K × K correlation matrix. By drawing M samples of size N one can compute M such correlation matrices, and one can determine the variances σ²_a,b of the sample of M correlation coefficients r_a,b, and the covariances σ_a,b;c,d between samples of size M of two different correlations r_a,b, r_c,d. For M → ∞, the elements of Ψ, which denotes the asymptotic variance-covariance matrix of the correlations times N, are given by [see Eqns (1) and (2) in Steiger (1980)]:

When two correlations have an index in common, the expression given for Ψ_a,b;c,d simplifies to [see Eq (3) in Steiger (1980)]:



If the raw sample correlations r_a,b are transformed by the Fisher r-to-z transform to z_a,b = 0.5 ln((1 + r_a,b)/(1 - r_a,b)), then the elements of the variance-covariance matrix times (N-3) of the transformed raw correlations are [see Eqs. (9)-(11) in Steiger (1980)]:

Test statistics

The test statistics proposed by Dunn and Clark (1969) are [see Eqns (12) and (13) in Steiger (1980)]:

where s_a,b;a,c and s_a,b;c,d denote sample estimates of the covariances c_a,b;a,c and c_a,b;c,d between the transformed correlations, respectively.

Note:

1. The SD of the difference of z_a,b − z_a,c given in the denominator of the formula for Z1⋆ depends on the value of ρ_a,b and ρ_a,c; the same holds analogously for Z2⋆.
2. The only difference between Z2⋆ and the z-statistic used for independent correlations is that in the latter the covariance s_a,b;c,d is assumed to be zero.

Central and noncentral distributions in power calculations

In the general case 'without a common index' the H₀ distribution is also the standard normal distribution N(0, 1). The H₁ distribution is the normal distribution N(m₁, s₁), with

s₁ =  √((2 − 2c₁)/(N − 3))/s₀, with c₁ = c_a,b;a,c for H₁,
m₁ = (z_a,b − z_c,d)/s₀, and
s₀ =  √((2 − 2c₀)/(N − 3)), with c₀ = c_a,b;c,d for H₀, i.e. ρ_a,b = ρ_c,d.

For the special case with a 'common index' the H₀ distribution is the standard normal distribution N(0, 1). The H₁ distribution is the normal distribution N(m₁ , s₁), with

s₁ =  √((2 − 2c₁)/(N − 3))/s₀, with c₁ = c_a,b;a,c for H₁,
m₁ = (z_a,b − z_a,c)/s₀, and
s₀ =  √((2 − 2c₀)/(N − 3)), with c₀ = c_a,b;a,c for H₀, i.e. ρ_a,b = ρ_a,c.

Validation

The results were checked against Monte-Carlo simulations.

References

Dunn, O. J., & Clark, V. (1969). Correlation coefficients measured on the same individuals. Journal of the American Statistical Association, 64, 366-377.

Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245-251.