Viewpoint: Why Disclosure Matters

Click to copy article linkArticle link copied!

Recent revelations about researchers failing to disclose industry funding of their work have raised the question: why does disclosure matter? An obvious answer is that if a journal has a disclosure policy, then failure to disclose violates that policy. But the issue is deeper than one of obeying rules; the important question is why those rules are necessary. The answer is that even if we think of ourselves as honest, objective, and independent, scientific evidence demonstrates that our research can be influenced by the sources of our funding.

A robust literature, dating to the mideighties, has documented this “funding effect.” Specifically, when funders have a particular desired outcome—that is, that tobacco smoking does not cause pancreatic cancer; that a particular chemical does not exhibit estrogenic activity; or that hydraulic fracturing for oil and gas development does not contaminate groundwater—the studies that industry funds are more likely to find that outcome than studies not so funded. The funding effect is particularly well documented in the domain of pharmaceuticals, where industry-funded studies have been shown to be significantly more likely to find outcomes favorable to the sponsors’ products—defined as greater efficacy or less harm for the sponsor’s product—than studies with other sources of funding. (1) The standard tools designed to prevent bias in clinical trials, such as blinding and randomization, do not prevent this effect.

Many of us are reluctant to accept this finding, because it seems to imply that our colleagues—including individuals we may know and respect—have been corrupted. Corruption exists, but the funding effect may more often be the result of unconscious bias. Researchers make many choices in the design, implementation, and interpretation of their work that involve expert judgment, and this opens a pathway through which unconscious bias may exert itself, both in study design and in data interpretation. In theory, such bias should be noticed in peer review; in practice, these subtleties often escape notice until results are contested post-publication by other researchers, re-examined in litigation, or detected in later reviews and meta-studies.

A well-documented example of this involves the choice of controls in experimental studies of suspected endocrine-disrupting chemicals. It may seem obvious that if the doses administered in an experiment are too low or the animal model is insensitive to the effect being studied, this can produce an inaccurate (false negative) result. Moreover, failure to establish the sensitivity of the animal model to the class of chemical being tested is a violation of the U.S. National Toxicology Program (NTP) recommendations for low-dose studies of endocrine disrupting chemicals. Yet several such flawed studies have been published in leading peer-reviewed journals. (2)

One reason scientists may succumb to unconscious bias is that we think ourselves less susceptible to these effects than we really are. A study of medical residents found that sixty-one percent argued that gifts from pharmaceutical companies would not affect their behavior, but thought that only 16 percent of their colleagues would remain similarly unaffected. (3) These results may be compared to the well-documented “third person effect,” in which people think that others are more influenced by advertising than they are. Scientists may be particularly susceptible to third person effects precisely because we think that we are not.

Moreover, many researchers have a narrow conception of research integrity, restricting it in their minds to avoiding egregious misconduct such as fraud, fabrication, and plagiarism. Many other behaviors can compromise research integrity, however, and evidence suggests that these behaviors may be widespread. One large, well-designed study, published in Nature in 2005, found that 33% of researchers admitted to questionable behaviors within the previous three years, including 20% of midcareer researchers who acknowledged “changing the design, methodology or results of a study in response to pressure from a funding source.” (4)

The Cochrane Reports, the leading source of systematic reviews in health care, recently concluded that the funding effect “is a known bias that should be assessed.” However, this is difficult to do on a case-by-case basis, because absent evidence of fraud, one cannot prove that a research result would have been different had the funders been different. The Cochrane researchers thus conclude that bias is best assessed “by using empirical methods to identify factors that are [systematically] associated with research results.” Such assessments of funding effects can only be performed if sources are known.

Scientists whose funding sources have been questioned have sometimes asserted that their analysis was not influenced by the source of their funding. (5) The problem is that they have no way to know that, and neither do those who rely on their results. This is why all journals should have disclosure policies and those policies should be enforced. But this raises the question: what is the means of enforcement? Journal editors should implement appropriate sanctions for violations of disclosure rules, up to and not excluding retraction.

Editors, reviewers, and readers make the default assumption that the research before them is unbiased; disclosure is essential because it alerts us to the fact that honest researchers may nevertheless be subject to unconscious bias, and that research integrity is not only threatened by a few “bad apples,” but by various forms of questionable practices that may be more widespread than most of us realize.