There is a lot of media attention these days to a supposed crisis in the replication of scientific studies. However, I don't agree that there is such a replication "crisis". Biological
experimentation involves a very high degree of complexity, with many
variables that are extremely difficult to control. For example, even the
same inbred mouse strains, or tissue culture cell lines, can
actually be quite divergent between different laboratories. It is very
difficult for one lab to assess the number of statistical degrees of
freedom in its own work. Thus it is to be expected that some, many even
many, results which are reported reliably on experiments performed in
good conscience, will fail to replicate among labs. That is part of the
cost of doing truly novel research. Just based on an anecdotal sense of
the kinds of work being described as "non-replicable", much of it is
either in the social sciences, where controlling variables is even
harder, or in clinical trials on human cohorts where again it is
challenging to control for environmental factors. In contrast, in my own field of
human medical genetics, results tend to be highly replicable, as we are
usually looking at the effects of severe high penetrance mutations.
Different mutations in the same gene typically generate very similar,
though not identical phenotypes even in very different populations. A
quick survey of the Human Gene Mutation Database corroborates this
observation.
That said, it is true that scientists are under way too much pressure to
generate positive results, especially for very expensive programs like
clinical trials or large scale population genetic studies. Moreover, the
hypercompetitive state of grant funding these days encourages hype and
excessive optimism over sober analysis. This is not the fault of the
NIH, this is the result of decades of government underfunding of science
measured as a proportion of national GDP.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment