Chi-Square Tests

There are two major categories of Chi2 tests: Goodness-of-fit-tests and Contingency tests. In both cases we have 2 distributions over m categories which are to be compared, one posited by H0 and one by H1. We use the effect size index w (Cohen, 1977). The noncentrality parameter lambda of the noncentral Chi2 distribution is given by   Lambda = w2 * N.

Goodness-of-Fit-Tests

H0 postulates a multinomial distribution across the m disjunct categories with probabilities p0(1), p0(2), ... , p0(m), with   m sum p0(i) = 1. i=1     H1 posits a different multinomial distribution with probabilities p1(1), p1(2), ... p1(m), with   m sum p1(i) = 1. i=1     The effect size index w is given by   ____________________________ / \ / m (p0(i) - p1(i))2 w = / sum _____________________ / i=1 p0(i) \/   You can easily calculate the effect size index w from these probabilities using the "Calc 'w'" option after you select "Type of Test: Chi-square Test." You will save time if you decide to use this option. The number of df (degrees of freedom) is m-1 if all probabilities have fixed values according to H0 and no parameter needs to be estimated. More df's can be lost in the process of parameter estimation. For instance, if the fit of empirical data to the normal distribution is tested, then 2 more df's are lost because the mean and the standard deviation have to be determined. Thus df = m - 1 - 2 in this case.

Example 1

We test how well an empirical distribution fits the normal distribution. First, we determine the theoretical probabilities for 10 intervals. We want to detect "small" deviations from the theoretical distribution according to Cohen's effect size conventions, thus w = 0.1. How many subjects do we need, given alpha = beta = .05?

Select:	Type of Power Analysis:	A Priori
	Type of Test:	Chi-square test
		Accuracy mode calculation

Input:	Alpha:	.05
	Power (1 - beta):	0.95
	Effect size "w":	0.100	To calculate conveniently the effect size from the probabilities defining H0 and H1, click "Calc 'w'", insert the probabilities and click "calc and copy")
	DF for Chi:	7	(m - 1 - 2 = 10 - 3 = 7)

Result:	Total sample size:	2184
	Actual power:	0.9500
	Critical Chi2:	Chi^2(7) = 14.0671
	Lambda:	21.8400

Example 2

       

Compromise power analyses can be of particular value when performing goodness-of-fit tests. For instance, it may be that we have very many data points such that, given alpha and beta = .05, even tiny and negligible deviations of the H0 and H1 probability distributions would result in rejections of the model. For instance, we could have 3500 data points and a 1 df model test, in which case the question would be which level of alpha = beta guarantees that only effects of at least w = 0.1 are detected. Let us suppose the relative seriousness of alpha and beta is given by the ratio q := beta/alpha = 1.

Select:	Type of Power Analysis:	Compromise
	Type of Test:	Chi-square test
		Accuracy mode calculation

Input:	Total sample size:	3500
	Effect size "w":	0.1000
	beta / alpha ratio:	1
	DF for Chi:	1

Result:	Alpha:	0.0022
	Power (1 - beta):	0.9978
	Critical Chi2:	Chi^2(1) = 9.3934
	Lambda:	35.0000

Contingency Tests

Suppose we have a two-dimensional I x J contingency table with i * j = m cells. H0 postulates that the random variables J and I are stochastically independent. In other words, the cell probabilities are determined by the associated column and row probabilities. H1, in contrast, posits that the distribution of the probabilities across the m cells is not determined by the column and row probabilities. Again, w is computed from the probability distributions according to H0 and H1 (see above). The degrees of freedom are given by             df = (i-1) * (j-1).

Example  

Let us test the independence assumption for a 2 x 3 table, that is, df = 1 * 2 = 2. Given a total sample size of 180, alpha = .05, and w = 0.327: What is the power of this test?

Select:	Type of Power Analysis:	Post hoc
	Type of Test:	Chi-square test
		Accuracy mode calculation

Input:	Alpha:	.05
	Effect size "w":	0.3270
	Total sample size:	180
	DF for Chi:	2

Result:	Power (1-beta):	0.9818
	Critical Chi^2:	Chi^2(2) = 5.9915
	Lambda:	19.2472

Please report suggestions for improvements to Axel Buchner, Franz Faul, or Edgar Erdfelder.