1997 OPEN FORUM Abstracts
How Can I Tell If What They Say Is True?
Joseph L. Rau, PhD, RRT, Tuesday, December 9, 1997.
The framework of a study is identified as incorporating multiple elements: assignment (selection & grouping), measurement, statistical analysis, study conclusion, and generalization beyond the study. Using these elements, a randomized controlled trial (RCT) is compared with less controlled studies using a case-control (retrospective) or a cohort (prospective) design. All three types of design are shown to investigate the effect of smoking on lung cancer, as an example. The logic of experimental research is intended to demonstrate that a change in a treatment (or cause) leads to a difference in outcome (effect). A general threat to any experimental study is the presence of nuisance or confounding variables. The internal validity of a study refers to the correctness of the conclusion about the treatment or intervention, and its effect on the outcome. Threats to internal validity include history, maturation, testing, instrumentation, group non-equivalence, loss of subjects, and low statistical power. The external validity of a study refers to the generalizability of the results to other settings and the target population. Experimenter and the "Hawthorn" effect are both threats, as well as lack of equivalence with the intended target population.
Each element of a study is examined for pitfalls, using a controlled trial design. Is a RCT feasible and ethical? Given that it is, is the group assignment random? Are treatment and measures double blind, and objective? Is the statistical analysis appropriate? Is the conclusion supported by the data? To what group(s) are the study results applicable?
An example of a hypothetical study is analyzed for possible flaws.
AARC 50th Anniversary, December 6 - 9, 1997, New Orleans, Louisiana.