Purpose: Case-control and observational studies are popular choices for evaluating molecular prognostic/pharmacogenetic outcomes, but data quality is rarely tested. Using clinical trial and epidemiologic methods, we assessed the quality of prognostic and outcomes data obtainable from a large case-control study of lung cancer.
Methods: We developed an explicit algorithm (set of standard operating procedures forming a rapid outcomes ascertainment system) that encompassed multiple tests of quality assurance, and quality of data for a range of prognostic and outcomes variables, in several cancers, across several centers and two countries were assessed. Based on these assessments, the algorithm was revised and physicians' clinical practice changed. We reevaluated the quality of outcomes after these revisions.
Results: Development of an algorithm with internal quality controls showed specific patterns of data collection errors, which were fixable. Although the major discrepancy rate in retrospective data collection was low (0.6%) when compared with external validated sources, complete data were found in <50% of patients for treatment response rate, toxicity, and documentation of patient palliative symptoms. Prospective data collection and changes to clinical practice led to significantly improved data quality. Complete data on response rate increased from 45% to 76% (P = 0.01, Fisher's exact test), for toxicity data, from 26% to 56% (P = 0.02), and for palliative symptoms, from 25% to 70% (P < 0.05), in one large lung cancer case-control study.
Conclusions: Observational studies can be a useful source for studying molecular prognostic and pharmacogenetic outcomes. A rapid outcomes ascertainment system with strict ongoing quality control measures is an excellent means of monitoring key variables.