There is a lot of media attention these days to a supposed crisis in the replication of scientific studies. However, I don't agree that there is such a replication "crisis". Biological experimentation involves a very high degree of complexity, with many variables that are extremely difficult to control. For example, even the same inbred mouse strains, or tissue culture cell lines, can actually be quite divergent between different laboratories. It is very difficult for one lab to assess the number of statistical degrees of freedom in its own work. Thus it is to be expected that some, many even many, results which are reported reliably on experiments performed in good conscience, will fail to replicate among labs. That is part of the cost of doing truly novel research. Just based on an anecdotal sense of the kinds of work being described as "non-replicable", much of it is either in the social sciences, where controlling variables is even harder, or in clinical trials on human cohorts where again it is challenging to control for environmental factors. In contrast, in my own field of human medical genetics, results tend to be highly replicable, as we are usually looking at the effects of severe high penetrance mutations. Different mutations in the same gene typically generate very similar, though not identical phenotypes even in very different populations. A quick survey of the Human Gene Mutation Database corroborates this observation.

That said, it is true that scientists are under way too much pressure to generate positive results, especially for very expensive programs like clinical trials or large scale population genetic studies. Moreover, the hypercompetitive state of grant funding these days encourages hype and excessive optimism over sober analysis. This is not the fault of the NIH, this is the result of decades of government underfunding of science measured as a proportion of national GDP.

What comes after

I woke this morning from a very interesting and (for me) atypical dream. Most dreams are of a private nature and publishing them nothing but an exercise in vanity, but this is perhaps one of those rare exceptions.

In the dream I was in the afterlife. No idea how I got there, that was irrelevant. It was not clearly depicted, but there many of "us", again not very clearly visualized. I seemed to be seated (?) at a console like a microfilm or microfiche machine, and images went speeding by as when one fast forwards through that type of film. The images were in color.

An authority, again not visualized, more like a voice (perhaps I was drawing on Tolkien's short story Leaf by Niggle), was explaining how things worked. Essentially all possible information was available, in other words everything that had ever happened (on earth? in the entire universe?), but it was up to each of us to focus our minds to narrow in on any information of interest to us. As a preliminary exercise, the voice suggested we solve a simple math puzzle, the answer to which was a number. I don't recall the number, but as I focused on it, the images I was seeing slowed, came into focus, and eventually settled on the correct number. As that was happening, I was also seeing images of children. The voice, now close to me, noted that before I arrived here (wherever here was), I had asked to see my grandchildren. They were among the children whose images had gone by, thus I had come near, but not perfectly, at obtaining my desired information. As it happens, earlier in the day (the real day, not the dream) I was leafing through Steichen's classic photoessay, Family of Man; perhaps the images of children in my dream were stimulated by that real-life activity. I am not aware of feeling strongly about grandchildren (none so far), perhaps it's more important to me than I realized.

Anyway, the voice continued that I had wanted to know something else, namely whether quantum mechanics and general relativity could ever be reconciled as a coherent theory of everything in physics. This is indeed something I've love to have solved by physicists (not me of course), and have often joked that that would be my first question on arriving in Heaven (or the other place, hopefully they both know!) The voice said that the answer was that the two theories could not be reconciled. I asked whether that meant both were incomplete approximations of a true mathematically consistent theory, which the voice confirmed.

Then I woke up. If only I had slept a few more moments, I might have had the answer to all physics! Probably not though, probably I woke up because that was as far as I could go. But I awoke with a sense of perfect peace, because the universe was about not struggle, but about knowledge and correct understanding, from which action and consequence flow naturally. This now seems rather buddhist to me, though most of the time I tend much more to activity, spontaneity and trial and error, versus deep thought. My wife must be changing me, or else I am changing myself in response to her wisdom.