Cronbach's alpha

By definition, reliability cannot be less than zero and cannot be greater than one. Many textbooks mistakenly equate ${\displaystyle \rho _{T}}$ with reliability and give an inaccurate explanation of its range. ${\displaystyle \rho _{T}}$ can be less than reliability when applied to data that are not essentially tau-equivalent. Suppose that ${\displaystyle X_{2}}$ copied the value of ${\displaystyle X_{1}}$ as it is, and ${\displaystyle X_{3}}$ copied by multiplying the value of ${\displaystyle X_{1}}$ by -1. The covariance matrix between items is as follows, ${\displaystyle \rho _{T}=-3}$.

Observed covariance matrix

	${\displaystyle X_{1}}$	${\displaystyle X_{2}}$	${\displaystyle X_{3}}$
${\displaystyle X_{1}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle -1}$
${\displaystyle X_{2}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle -1}$
${\displaystyle X_{3}}$	${\displaystyle -1}$	${\displaystyle -1}$	${\displaystyle 1}$

Negative ${\displaystyle \rho _{T}}$ can occur for reasons such as negative discrimination or mistakes in processing reversely scored items.

Unlike ${\displaystyle \rho _{T}}$, SEM-based reliability coefficients (e.g., ${\displaystyle \rho _{C}}$) are always greater than or equal to zero.

This anomaly was first pointed out by Cronbach (1943)[20] to criticize ${\displaystyle \rho _{T}}$, but Cronbach (1951)[10] did not comment on this problem in his article that otherwise discussed potentially problematic issues related ${\displaystyle \rho _{T}}$.[9]^: 396

This anomaly also originates from the fact that ${\displaystyle \rho _{T}}$ underestimates reliability. Suppose that ${\displaystyle X_{2}}$ copied the value of ${\displaystyle X_{1}}$ as it is, and ${\displaystyle X_{3}}$ copied by multiplying the value of ${\displaystyle X_{1}}$ by two. The covariance matrix between items is as follows, ${\displaystyle \rho _{T}=0.9375}$.

Observed covariance matrix

	${\displaystyle X_{1}}$	${\displaystyle X_{2}}$	${\displaystyle X_{3}}$
${\displaystyle X_{1}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 2}$
${\displaystyle X_{2}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 2}$
${\displaystyle X_{3}}$	${\displaystyle 2}$	${\displaystyle 2}$	${\displaystyle 4}$

For the above data, both ${\displaystyle \rho _{P}}$ and ${\displaystyle \rho _{C}}$ have a value of one.

The above example is presented by Cho and Kim (2015).[7]

Many textbooks refer to ${\displaystyle \rho _{T}}$ as an indicator of homogeneity[21] between items. This misconception stems from the inaccurate explanation of Cronbach (1951)[10] that high ${\displaystyle \rho _{T}}$ values show homogeneity between the items. Homogeneity is a term that is rarely used in the modern literature, and related studies interpret the term as referring to uni-dimensionality. Several studies have provided proofs or counterexamples that high ${\displaystyle \rho _{T}}$ values do not indicate uni-dimensionality.[22][7][23][24][25][26] See counterexamples below.

Uni-dimensional data

	${\displaystyle X_{1}}$	${\displaystyle X_{2}}$	${\displaystyle X_{3}}$	${\displaystyle X_{4}}$	${\displaystyle X_{5}}$	${\displaystyle X_{6}}$
${\displaystyle X_{1}}$	${\displaystyle 10}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$
${\displaystyle X_{2}}$	${\displaystyle 3}$	${\displaystyle 10}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$
${\displaystyle X_{3}}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 10}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$
${\displaystyle X_{4}}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 10}$	${\displaystyle 3}$	${\displaystyle 3}$
${\displaystyle X_{5}}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 10}$	${\displaystyle 3}$
${\displaystyle X_{6}}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 3}$	${\displaystyle 10}$

${\displaystyle \rho _{T}=0.72}$ in the uni-dimensional data above.

Multidimensional data

	${\displaystyle X_{1}}$	${\displaystyle X_{2}}$	${\displaystyle X_{3}}$	${\displaystyle X_{4}}$	${\displaystyle X_{5}}$	${\displaystyle X_{6}}$
${\displaystyle X_{1}}$	${\displaystyle 10}$	${\displaystyle 6}$	${\displaystyle 6}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$
${\displaystyle X_{2}}$	${\displaystyle 6}$	${\displaystyle 10}$	${\displaystyle 6}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$
${\displaystyle X_{3}}$	${\displaystyle 6}$	${\displaystyle 6}$	${\displaystyle 10}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$
${\displaystyle X_{4}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 10}$	${\displaystyle 6}$	${\displaystyle 6}$
${\displaystyle X_{5}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 6}$	${\displaystyle 10}$	${\displaystyle 6}$
${\displaystyle X_{6}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 6}$	${\displaystyle 6}$	${\displaystyle 10}$

${\displaystyle \rho _{T}=0.72}$ in the multidimensional data above.

Multidimensional data with extremely high reliability

	${\displaystyle X_{1}}$	${\displaystyle X_{2}}$	${\displaystyle X_{3}}$	${\displaystyle X_{4}}$	${\displaystyle X_{5}}$	${\displaystyle X_{6}}$
${\displaystyle X_{1}}$	${\displaystyle 10}$	${\displaystyle 9}$	${\displaystyle 9}$	${\displaystyle 8}$	${\displaystyle 8}$	${\displaystyle 8}$
${\displaystyle X_{2}}$	${\displaystyle 9}$	${\displaystyle 10}$	${\displaystyle 9}$	${\displaystyle 8}$	${\displaystyle 8}$	${\displaystyle 8}$
${\displaystyle X_{3}}$	${\displaystyle 9}$	${\displaystyle 9}$	${\displaystyle 10}$	${\displaystyle 8}$	${\displaystyle 8}$	${\displaystyle 8}$
${\displaystyle X_{4}}$	${\displaystyle 8}$	${\displaystyle 8}$	${\displaystyle 8}$	${\displaystyle 10}$	${\displaystyle 9}$	${\displaystyle 9}$
${\displaystyle X_{5}}$	${\displaystyle 8}$	${\displaystyle 8}$	${\displaystyle 8}$	${\displaystyle 9}$	${\displaystyle 10}$	${\displaystyle 9}$
${\displaystyle X_{6}}$	${\displaystyle 8}$	${\displaystyle 8}$	${\displaystyle 8}$	${\displaystyle 9}$	${\displaystyle 9}$	${\displaystyle 10}$

The above data have ${\displaystyle \rho _{T}=0.9692}$, but are multidimensional.

Uni-dimensional data with unacceptably low reliability

	${\displaystyle X_{1}}$	${\displaystyle X_{2}}$	${\displaystyle X_{3}}$	${\displaystyle X_{4}}$	${\displaystyle X_{5}}$	${\displaystyle X_{6}}$
${\displaystyle X_{1}}$	${\displaystyle 10}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$
${\displaystyle X_{2}}$	${\displaystyle 1}$	${\displaystyle 10}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$
${\displaystyle X_{3}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 10}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$
${\displaystyle X_{4}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 10}$	${\displaystyle 1}$	${\displaystyle 1}$
${\displaystyle X_{5}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 10}$	${\displaystyle 1}$
${\displaystyle X_{6}}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 1}$	${\displaystyle 10}$

The above data have ${\displaystyle \rho _{T}=0.4}$, but are uni-dimensional.

Uni-dimensionality is a prerequisite for ${\displaystyle \rho _{T}}$. One should check uni-dimensionality before calculating ${\displaystyle \rho _{T}}$ rather than calculating ${\displaystyle \rho _{T}}$ to check uni-dimensionality.[3]

The term internal consistency is commonly used in the reliability literature, but its meaning is not clearly defined. The term is sometimes used to refer to a certain kind of reliability (e.g., internal consistency reliability), but it is unclear exactly which reliability coefficients are included here, in addition to ${\displaystyle \rho _{T}}$. Cronbach (1951)[10] used the term in several senses without an explicit definition. Cho and Kim (2015)[7] showed that is ${\displaystyle \rho _{T}}$ is not an indicator of any of these.

Removing an item using "alpha if item deleted" may result in 'alpha inflation,' where sample-level reliability is reported to be higher than population-level reliability.[27] It may also reduce population-level reliability.[28] The elimination of less-reliable items should be based not only on a statistical basis but also on a theoretical and logical basis. It is also recommended that the whole sample be divided into two and cross-validated.[27]

Nunnally's book[29][30][31] is the most commonly mentioned source for how high dependability coefficients should be. However, his proposals contradict his aims. He meant that different criteria should be used based on the goal or stage of the investigation. Regardless of the study kind, such as exploratory research, applied research, or scale development research, a criterion of 0.7 is universally used.[32] He advocated 0.7 as a criterion for early stages of a study, which most studies published in the journal are not. Rather than 0.7, Nunnally's applied research criterion of 0.8 is more suited for most empirical studies.[32]

His recommendation level did not imply a cutoff point. If a criterion means a cutoff point, it is important whether or not it is met, but it is unimportant how much it is over or under. He did not mean that it should be strictly 0.8 when referring to the criteria of 0.8. If the reliability has a value near 0.8 (e.g., 0.78), it can be considered that his recommendation has been met.[33]

Nunnally's idea was that there is a cost to increasing reliability, so there is no need to try to obtain maximum reliability in every situation.

Measurements with perfect reliability lack validity.[7] For example, a person who take the test with the reliability of one will get a perfect score or a zero score, because the examine who gives the correct answer or incorrect answer on one item will give the correct answer or incorrect answer on all other items. The phenomenon in which validity is sacrificed to increase reliability is called attenuation paradox.[34][35]

A high value of reliability can be in conflict with content validity. For high content validity, each item should be constructed to be able to comprehensively represent the content to be measured. However, a strategy of repeatedly measuring essentially the same question in different ways is often used only for the purpose of increasing reliability.[36][37]

When the other conditions are equal, reliability increases as the number of items increases. However, the increase in the number of items hinders the efficiency of measurements.

Despite the costs associated with increasing reliability discussed above, a high level of reliability may be required. The following methods can be considered to increase reliability.

Before data collection:

• Eliminate the ambiguity of the measurement item.
• Do not measure what the respondents do not know.[38]
• Increase the number of items. However, care should be taken not to excessively inhibit the efficiency of the measurement.
• Use a scale that is known to be highly reliable.[39]
• Conduct a pretest - discover in advance the problem of reliability.
• Exclude or modify items that are different in content or form from other items (e.g., reverse-scored items).

After data collection:

• Remove the problematic items using "alpha if item deleted". However, this deletion should be accompanied by a theoretical rationale.
• Use a more accurate reliability coefficient than ${\displaystyle \rho _{T}}$. For example, ${\displaystyle \rho _{C}}$ is 0.02 larger than ${\displaystyle \rho _{T}}$ on average.[40]

Existing studies are practically unanimous in that they oppose the widespread practice of using ${\displaystyle \rho _{T}}$ unconditionally for all data. However, different opinions are given on which reliability coefficient should be used instead of ${\displaystyle \rho _{T}}$.

Different reliability coefficients ranked first in each simulation study[41][42][6][43][44] comparing the accuracy of several reliability coefficients.[7]

The majority opinion is to use SEM-based reliability coefficients as an alternative to ${\displaystyle \rho _{T}}$.[3][7][45][5][46][8][6][47]

However, there is no consensus on which of the several SEM-based reliability coefficients (e.g., uni-dimensional or multidimensional models) is the best to use.

Some people suggest ${\displaystyle \omega _{H}}$[6] as an alternative, but ${\displaystyle \omega _{H}}$ shows information that is completely different from reliability. ${\displaystyle \omega _{H}}$ is a type of coefficient comparable to Reveille's ${\displaystyle \beta }$.[48][6] They do not substitute, but complement reliability.[3]

Among SEM-based reliability coefficients, multidimensional reliability coefficients are rarely used, and the most commonly used is ${\displaystyle \rho _{C}}$,[3] also known as composite or congeneric reliability.

General-purpose statistical software such as SPSS and SAS include a function to calculate ${\displaystyle \rho _{T}}$. Users who don't know the formula of ${\displaystyle \rho _{T}}$ have no problem in obtaining the estimates with just a few mouse clicks.

SEM software such as AMOS, LISREL, and MPLUS does not have a function to calculate SEM-based reliability coefficients. Users need to calculate the result by inputting it to the formula. To avoid this inconvenience and possible error, even studies reporting the use of SEM rely on ${\displaystyle \rho _{T}}$ instead of SEM-based reliability coefficients.[3] There are a few alternatives to automatically calculate SEM-based reliability coefficients.

1. R (free): The psych package[49] calculates various reliability coefficients.
2. EQS (paid):[50] This SEM software has a function to calculate reliability coefficients.
3. RelCalc (free):[3] Available with Microsoft Excel. ${\displaystyle \rho _{C}}$ can be obtained without the need for SEM software. Various multidimensional SEM reliability coefficients and various types of ${\displaystyle \omega _{H}}$ can be calculated based on the results of SEM software.