A recent story in the Chronicle of Higher Education suggests that Michael LaCour, a UCLA graduate student, fabricated data for a journal article. Science magazine has retracted the article, due to “the misrepresentation of survey incentives, the false sponsorship statement, and Mr LaCour’s inability to produce original data”.
This is only the latest in a long string of integrity issues in research publications. The scary reality, though, is not that articles have been found to be questionable, but that it is possible that many other fabricated articles have not been discovered and retracted.
Meanwhile, John Bohannon, a biologist and science journalist at Harvard University, intentionally published some weak and questionable findings related to chocolate just to demonstrate how quickly non-refereed journals would snatch up research. He claims: “I fooled millions into thinking chocolate helps weight loss.”
Sign me up!
In the United States, the Office of Research Integrity oversees integrity on behalf of the Secretary of Health and Human Services. It is currently investigating 50 cases of research misconduct.
Granted, the vast majority of published research is carefully reviewed and published with full integrity (we think; we hope). Nevertheless, one should be properly skeptical of the scientific claims. How can we be more informed consumers of research claims?
There is some comfort in that in the internet age it is getting harder to succeed in research fraud. The ease of communication, the requirements of documentation, and the focus on accountability should strongly communicate that research fraud will ruin careers.
However, consumers should realise that not all journals are created equal. The best and most trustworthy journals are peer reviewed and have a low acceptance and publication rate. By comparison, there is reason to be more cautious about fee-charging open access journals. Nonetheless, it should be noted that the LaCour article mentioned above was published in a highly regarded peer-reviewed journal. Even the best journals sometimes make the wrong decision.
Most importantly, consumers should look for replication. Scientific findings that are reported by only one researcher or one laboratory should be considered to be only working hypotheses. Our confidence level should increase only when findings are replicated by others.
The 1989 claim of cold fusion is a case in point. This exciting claim caught the attention of the scientific world, but those who sought to replicate the findings were not able to do so. Indeed, good science requires replication. When multiple researchers in multiple labs using multiple methods reach similar conclusions, we have more confidence in their accuracy. Replication provides the best protection against scientific fraud.
Ultimately, it is always wise and prudent to be skeptical about scientific research. Good science gives us valuable new insights into the world in which we live. Nevertheless, even when properly conducted, science does not yield “proof”. The testing of hypotheses is based on statistics and probabilities.
Scientific research always includes some probability that the findings resulted from random noise rather than systematic effects. We can sometimes reduce the probability of reaching false conclusions to 1 per cent, or 0.1 per cent, or even 0.000001 per cent, but we can never reduce the probability to zero. Anything more than zero means that we have not (indeed we cannot) achieve statistical “proof”. The words “prove” and “proof” are valuable and appropriate in the field of mathematics. They are not, however, acceptable in research terminology. Good research seeks to provide new evidence; it does not seek to achieve “proof”.
Because there is always a non-zero chance of error, it is always wise to be skeptical of science. Indeed, the skeptic is the individual who asks the “what if” questions that enable the research to move to the next level.
The scientific method is a most valuable tool to help us understand how the world works. Yes, there is some risk that individuals might seek to publish fraudulent reports as a means to advance their career. There is some risk that individuals might seek to publish fraudulent reports as a means to advance their agenda. As a protection, however, good science always calls for replication.
Simple replication of previous research has in the past often been viewed negatively. The best scientist is always trying to do something new and different in order to advance the field. In fact, however, simple replication of previous research is more important than ever, as a means of validating the integrity of research.
Although there is no way to prove the accuracy of any study (but of course, “prove” is not a scientific word), our pursuit of knowledge requires good research efforts. Part of our ongoing search for knowledge must be to be forever on our guard against fraud.
Dr Gary Welton is assistant dean for institutional assessment, professor of psychology at Grove City College in Pennsylvania, U.S., and a contributor to The Center for Vision & Values.