Spectrum bias

Tests do not perform equally in all populations. A chest x ray is very accurate in detecting large tension pneumothoraces in sick patients, but will miss tiny (clinically questionable) pneumothoraces in healthy patients. The accuracy of a diagnostic test depends on the severity and prevalence of the disease in the population being studied. If a study’s population is significantly different from the population the diagnostic test will ultimately be used on, the numbers will be inaccurate, and the resulting bias is called spectrum bias.

Using a test in a population who obviously do not have a disease will artificially improve the test’s sensitivity.

Using a test in a population who obviously do have a disease will artificially improve the test’s specificity.

Although theoretically sensitivity and specificity will remain constant as a disease prevalence changes, in reality this assumption frequently fails. The reason is that as a disease’s prevalence changes, so does its severity, and the severity of disease has a significant impact on the sensitivity and specificity of the tests we use to diagnose it.

For example, rheumatic arthritis is rare in family doctors’ offices, but relatively common in the offices of rheumatologists. This shift in prevalence should not affect the specificity of a test like hand inspection for joint deformity. However, the rheumatologists are also seeing sicker patients, which means the test is actually more specific in their hands.

This post is part of a series of posts on bias in medical research. You can find the whole bias catalogue here.

You can find more evidence based medicine resources here.

References

Montori VM, Wyer P, Newman TB, Keitz S, Guyatt G. Tips for learners of evidence-based medicine: 5. The effect of spectrum of disease on the performance of diagnostic tests. CMAJ. 2005; 173(4):385-90. PMID: 16103513 [free full text]