Proportions: Inequality, two dependent groups (McNemar)

This procedure relates to tests of paired binary responses. Such data can be represented in a 2 × 2 table:

	Standard
Treatment	Yes	No
Yes	π11 π12 π21 π22		πt
No			1-πt
	πs	1-πs	1

where π_ij denotes the probability of the respective response. The probability π_D of discordant pairs, that is, the probability of yes/no-responses pairs, is given by π_D = π₁₂ + π₂₁. The hypothesis of interest is that π_s = π_t , which is formally identical to the statement π₁₂ = π₂₁.

Using this fact, the null hypothesis states (in a ratio notation) that π₁₂ is identical to π₂₁, and the alternative hypothesis states that π₁₂ and π₂₁ are different:

H₀ : π₁₂ /π₂₁ = 1
H₁ : π₁₂ /π₂₁ ≠ 1

 
In the context of the McNemar test the term odds ratio (OR) denotes the ratio π₁₂/π₂₁ that is used in the formulation of H₀  and H₁.

Effect size index

The odds ratio π₁₂/π₂₁ is used to specify the effect size. The odds ratio must lie inside the interval [10⁻⁶, 10⁶]. An odds ratio of 1 corresponds to a null effect. Therefore, this value must not be used in a priori power analyses.

In addition to the odds ratio, the proportion of discordant pairs, i.e. π_D, must be provided in the input parameter field called Prop discordant pairs. The values for this proportion must lie inside the interval [ε, 1 − ε], with ε = 10⁻⁶.
 
If π_D and d = π₁₂ - π₂₁ are given, then the odds ratio may be calculated as OR = (d + π_D)/(d − π_D).

Options

Alpha balancing in two-sided tests

The binomial distribution is discrete. It is therefore not normally possible to arrive at the exact nominal α-level. For two-sided tests this leads to the problem of how to "distribute" α to the two sides. G*Power offers the three options listed here, the first option being selected by default:

1. Assign α/2 to both sides: Both sides are handled independently in exactly the same way as in a one-sided test. The only difference is that α/2 is used instead of α. Of the three options offered by G*Power, this one leads to the largest deviation from the actual α (in post hoc analyses).
   
   
2. Assign to minor tail α/2, then rest to major tail (α2 = α/2, α1 = α − α2 ): First α/2 is applied to the side of the central distribution that is farther away from the noncentral distribution (minor tail). The criterion used for the other side is then α − α₁, where α₁ is the actual α found on the minor side. Since α₁ ≤ α/₂ one can conclude that (in post hoc analyses) the sum of the actual values α₁ + α₂ is in general closer to the nominal α-level than it would be if α/2 were assigned to both sides (see Option 1).
   
   
3. Assign α/2 to both sides, then increase to minimize the difference of α1 + α2 to α: The first step is exactly the same as in Option 1. Then, in the second step, the critical values on both sides of the distribution are increased (using the lower of the two potential incremental α-values) until the sum of both actual α values is as close as possible to the nominal α.

Press the Options button in the main window to select one of these options.

Computation

You may choose between an exact procedure and a faster approximation (see implementation notes for details):

1. Exact (unconditional) power if N < x. The computation time of the exact procedure increases much faster with sample size N than that of the approximation. Given that both procedures usually produce very similar results for large sample sizes, a threshold value x for N can be specified which determines the transition between both procedures. The exact procedure is used if N < x; the approximation is used otherwise. Note: G*Power does not show distribution plots for exact computations.
   
2. Faster approximation (assumes number of discordant pairs to be constant). Choosing his option instructs G*Power always to use the approximation.

Examples

As an example we replicate the computations in O'Brien (2002, p. 161-163). The assumed table is:

	Standard
Treatment	Yes	No
Yes	.54 .08 .32 .06		.62
No			.38
	.86	.14

In this table the proportion of discordant pairs is π_D = .32 + .08 = 0.4 and the Odds Ratio OR = π₁₂/π₂₁ = 0.08/.32 = 0.25. We want to compute the exact power for a one-sided test. The sample size N, that is, the number of pairs, is 50 and α = 0.05.

Select

Type of power analysis: Post hoc

Options

Computation: Exact

Input

Tail(s): One
Odds ratio: 0.25
α err prob: 0.05
Total sample size: 50
Prop discordant pairs: 0.4

Output

Power (1-β err prob): 0.839343
Actual α: 0.032578
Proportion p12: 0.08
Proportion p21: 0.32

The  power calculated by G*Power (0.839343) corresponds within the given precision to the result computed by O'Brien (0.839). Now we use the Power Plot window to calculate the power for several other sample sizes and also to generate a graph that gives us an overview of a section of the parameter space. The Power Plot window can be opened by pressing the X-Y plot for a range of values button in the lower part of the main window

In the Power Plot window we choose to plot the power on the Y axis (with markers and displaying the values in the plot) as a function of the total sample size. The total sample sizes shall range from 50 in steps of 25 trough to 150. We choose to draw a single plot. We specify α = 0.05 and odds ratio = 0.25.

The results shown in the figure below replicate exactly the values in the table shown in O'Brien (2002) on p. 163.



In order to replicate the values for the two-sided case, we must decide how the α error should be distributed to the two sides. The method chosen by O'Brien (2002) corresponds to Option 2 in G*Power (Assign to minor tail α/2, then rest to major tail, see above). In the main window, we need to select Tail(s): Two and set the other input parameters exactly as shown in the example above. For sample sizes of 50, 75, 100, 125, and 150 we get power values of 0.798241, 0.930639, 0.980441, 0.994839, and 0.998658, respectively, which are again equal to the values given in O'Brien's table.

Related tests

Proportions: Inequality, two independent groups (Fisher's exact test)

Implementation notes

Exact (unconditional) power if N < x

In this case G*Power calculates the unconditional power for the exact conditional test. The number of discordant pairs N_D is a random variable with binomial distribution B(N, π_D), where N denotes the total number of pairs, and π_D = π₁₂ + π₂₁ denotes the probability of discordant pairs. Thus P(N_D) =  (binomial(N, N_D)) · (π₁₁ + π₂₂ )^N_D · (π₁₂ + π₂₁)^N−N_D. Conditional on N_D, the frequency f₁₂ has a binomial distribution B(N_D , π₀ = π₁₂ /π_D) and we test the H₀: π₀ = 0.5. Given the conditional 'binomial power' Pow(N_D , π₀|N_D = i) the exact unconditional power is ∑_i^N P(N_D = i)Pow(N_D, π₀ |N_D = i). The summation starts at the most probable value for N_D and then steps outward until the values are small enough to be ignored.

Fast approximation

In this case an ordinary one sample binomial power calculation is performed with H₀ distribution B(Nπ_D, 0.5) and H₁ distribution B(Nπ_D, OR/(OR + 1)).

Validation

The results of the exact procedure were checked against the values given on pages 161-163 in O'Brien (2002). Complete correspondence was found in the one-tailed case and also in the two-tailed case when the alpha balancing Option 2 (Assign to minor tail α/2, then rest to major tail, see above) was chosen in G*Power.
 
We also compared the exact results of G*Power generated for a large range of parameters to the results produced by PASS (Hintze, 2006) for the same scenarios. We found complete correspondence in one-sided tests. In two-sided tests PASS uses an alpha balancing strategy corresponding to Option 1 in G*Power (Assign α/2 on both sides, see above). With two-sided tests we found small deviations between G*Power and PASS (about ±1 in the third decimal place), especially for small sample sizes. These deviations were always much smaller than those resulting from a change of the balancing strategy. All comparisons with PASS were restricted to N < 2000, because for larger N the exact routine in PASS sometimes produced nonsensical values (this restriction is noted in the PASS manual).

References

Cohen, J. (1969). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.
Hintze, J. (2006). NCSS, PASS, and GESS. Kaysville, Utah: NCSS.
O'Brien, R. (2002). Sample size analysis in study planning (using unifypow.sas). (available on the WWW:
http://www.bio.ri.ccf.org/UnifyPow.all/UnifyPowNotes020811.pdf )