Guides to Use of Medical Literature

In analyzing the results of a published medical study, you must have confidence that the results of the study are valid - i.e. that they conclusions follow directly from the observations - and that you understand the results of a treatment.   A short synopsis of two articles on medical studies published in the Journal of the American Medical Society (JAMA) is given below.  The two articles are:

Are the results of the study valid?:     Do the results represent an unbiased estimate of a treatment's positive effects or of a potential toxic effect of exposure or have they been influenced in some systematic fashion to lead to a false conclusion?  Several factors must be considered?

A.  Where there clearly identified comparison groups (experimental vs. control) that were similar with respect to important determinants of Outcomes other than the one of interest - i.e. the dependent variable?   Three kinds of trials have been used in such assessments:

  1. Randomized Control Trials (RCT) - An experiment in which patients are assigned in a random fashion to the experimental group (i.e.   given a treatment of unknown efficacy) or control groups (i.e. for a treatment study, given a placebo or standard therapy).  RCT are rarely used for harm studies.   The power of RCT stems from the fact that patients are randomly selected which raises the probability that experimental and control groups are identical except for the experimental treatment. 
  2. Cohort Studies - Used when it is impossible or unethical to perform a RCT.  In this study the investigators identifies two groups of people:  one using a particular treatment or exposed to a potentially harmful agent and another not using the treatment or not exposed.  The participants are followed to determine difference in effects of treatment or exposure to harm with time.  An example would be to select men (or women)  exposed to toxins in a given industry and assess the health status of their children.  Alternatively, cohort studies can be done when the outcome is infrequent.  Since the people in the study are not randomly picked (since they pick themselves or are picked by doctors), members of the exposed group may be different in other characteristics  from the control.  Investigators must show that the groups are similar (except with respect to the dependent variable) or use statistics to adjust for any differences.  (For example, a study for GI bleeding in people who take non-steroidal anti-inflammatory drugs (NSAID) (like tylenol, advil, aspirin) should be controlled for age, since increased age as well as increased use of NSAID are associated with GI bleeding.)  Even then other confounding variables might be associated with the effect, such as if illness for which people use NSAID might cause GI bleeding
  3. Case Control - Used if outcome very rare or takes long to develop.  Investigators select people  with specific outcome and then chooses controls without that outcome but as similar in other ways to experiment group (ex:  identical in age, sex, weight, etc). then ascertain past exposure to therapy or toxins in both groups.  (For example, show association between DES intake by pregnant women and development of cancer in their daughters. 
  4. Case Reports - Do not use control groups and are merely descriptive. (not a trial)

B.  Were the exposures to potentially toxic agent and the outcomes (of RCT and cohort studies)  measured the same in both experimental and control groups?  For example, in a case control study, when asked about prior exposure to solvents, patients in the experimental group (with leukemia) might be more likely to recall and report previous exposure because of greater motivation (recall bias) or greater probing by an interviewer (interviewer bias).   Exposures to potentially toxic agents should also be for a similar length.   In a cohort study, a three-fold increase risk of cancer in radiation workers might be explained by a physician who was aware of the risk doing a more thorough assessment of the patient which might detect disease earlier or disease that would ordinarily go undetected (surveillance bias). 

C. Were the patients available for follow-up for a sufficiently long enough period of time?  If patients are not available for follow-up, the validity of the study might be compromised since that that remain available might have different prognosis than those who aren't available. 

D. Were patients analyzed in the groups to which they were randomized?  If someone in a experimental group taking a drug become noncompliant and don't take the medication, they should not be excluded from the analyses.   Often these patients fared worse.  Excluding them (while leaving similar patients in the control group) leaves behind those who might have gotten better, and removes the unbiased nature of the RCT. 

E.  Were patients, clinicians, and study personnel "blind" to treatment?  If a patient knows they are on a new therapy, they will have an opinion about its efficacy, as will the practitioner and others involved in the study.  The opinions can systematically distort aspects of the treatment, reporting of outcomes, and introduce bias of the medical personal into the assessment of the treatment efficacy.  Best way to avoid is performing a double-blind study (for drug studies, using a placebo  for the control group and target drug for the experimental group).

 

You should be able to differentiate among these methods, and understand the nature of blind and control studies, as well as the various biases.