Jerry Coyne blogged yesterday about the trend in articles pointing out the flaws in science, noting that most of the observed problems are in medical studies, most notably in drug studies, and that generalizing these problems to all of science isn’t really accurate or fair.
I agree, but I have an observation about why some fields have problems, and other don’t. The natural sciences, such as physics, chemistry, biology, and geology don’t seem to be having particular problems. Physicists are having a debate about whether certain theoretical concepts are really science, but that is a minor and relatively healthy debate, compared to the issues that these articles are discussing.
Issues such as unrepeatable results, shoddy methodologies, selective publication of data, and ideologically driven interpretations seem to predominate in certain fields, but not others. Initially, I thought maybe it was just that fields with politically sensitive topics were the problem ones, but biology, despite much of society’s antipathy toward evolution, doesn’t seem to have these issues.
But some fields seem to have unusual problems in this area. Economics comes to mind. It’s a field where building a consensus appears to be a major challenge. A lot of people point to the fact that economists can’t do experiments, that they can only observe what happens and try to build theories based on those observations.
Perhaps, but I think the bigger problem is that a lot of financially powerful constituencies have a stake in the results of economic studies, and that has a major influence on the profession. There is a science of economics, and Paul Krugman and others often represent the voice of that science. Krugman often calls for more empirical work, which bizarrely is something many economists resist, to the profession’s discredit as a science.
The other field that stands out is pharmacology, the study of drug effects and interactions. Of course, the drug industry is heavily vested in the results of studies in this field, and actually probably fund most of it. As the linked articles point out, most of the results in these studies can’t be replicated.
But politics and money can’t always damage a field. Despite ferocious pushback from industry, the climatology field has largely stuck to its guns on climate change. Although the conflict between this field and industry is relatively new, so it remains to be seen how well it will endure over the long term.
So, it seems to me that fields with subjects that are not considered controversial by powerful constituencies, have a higher level of credibility. But where money and politics comes into the picture, the science starts to become influenced to an unhealthy degree.
I’m not sure what the solution to this is, other than to be cognizant of who funded a study, and whether or not its results have been replicated.