“A lot of people simply won’t read a 15-page whitepaper, but will be impressed by flowcharts. By making the language of Web3 meandering and impenetrable and by building a culture that is very self-referential, investors make criticism harder to come by.”

Today’s Galaxy Brain newsletter is about Web3, but replace “whitepaper” with “manuscript” and “investors” with “researchers” and you get bad science in a nutshell.

“If you are trying to figure out a thinker and his or her defects, see if you can spot that person’s “once-and-for-all” moves. There will be plenty of them.”

Cowen is right, though we can debate whether early closure is a defect or a feature.

“This contains a combination of various life-extension medicines (metformin, ashwagandha, and some vitamins), and covid defense gear: a CO2 meter… masks, antigen tests and fluvoxamine.”

The reddest of crypto world’s red flags is their belief in longer life through chemistry.

“The convergence of genomics of the cancer—be it from the person’s DNA or tumor directly or the blood (known as liquid biopsy)—matched with the appropriate therapy is leading to outcomes that are being described as ‘unheard-of’ by expert oncologists.”

So writes one Eric Topol, who seems to have made a career out of telling high-status people what they want to hear. For reasons why most of what he wrote is wrong, take a few minutes to watch Vinay Prasad’s reviews of the articles in question.

The trend disrupting medicine back when Dr. Topol was writing about its Creative Destruction turned out to be not technology but rather the opioid epidemic. As a non-expert oncologist I would wager something other than liquid biopsies is leaving its hallmark on the field right now.

“The speed of review times and increasing number of FDA-approved cancer medicines has long been used as a metric for successful regulatory processes and improvements in patient outcomes.”

Here are three good reasons why that isn’t so, to which I would add Goodhart’s law.

The NYT dostarlimab article is reverberating through international media with predictable consequences: being hailed as a miracle cure for cancer. I wrote about it in Serbian, and Google’s translation of it is readable, in an AI-generated spam sort of way.

Residency programs' publication requirements generate the worst kind of scientific BS — useless case reports and rushed chart reviews that clog lit searches without creating any meaningful insight, all while wasting residents' precious time. Please stop.

“Brilliance represents an upper bound on the quality of your reasoning, but there is no lower bound.”

A frameable quote if there ever was one, from an excellent blog post on why smart people may believe dumb things.

“At a conference a few months before the pandemic, a scholar told me how, in his department, everyone wrote lengthy pre-analysis plans that would, in theory, constrain P-hacking. In practice, he admitted, researchers could give cherry picking free rein, counting on the fact that no one has the time or patience to read a 100-page pre-analysis plan and compare it with the later publication.”

This is from an opinion piece in Nature about reproducibility of science in general, but the sentiment holds for clinical research (arguably, clinical research is even worse, for those hailing results of a clinical trial as game-changing often can’t be bothered with reviewing even a brief ClinicalTrials.gov registration and its history of changes, let alone a 100-page document). You could argue that open analysis plans at least give authors a reason to fear, as a deviation from the plan could result in negative letters to the Editor, retractions, or worse yet, shaming on social networks. In practice, it is those who call out deviations who are more afraid of name calling, Twitter mobs, and employer @-mentions for… being too aggressive, I guess?

Once you incentivize scientists to produce hundreds of pages of text that no human will ever read and no algorithm will parse just so a box or two can be checked, do not be surprised at the outcome.

You don’t have to convince me that NIH grant funding needs to be reformed, but all this article in The Atlantic did was show that even people who get their funding from NIH (i.e. the two authors) have no clue about how NIH operates or what its mission is.

For instance, this assertion:

“… the NIH, the largest public funder of clinical trials in the United States, should also have been well positioned to create treatment guidance for doctors caring for patients hospitalized with a brand-new disease.”

is followed shortly by:

“Moreover, the NIH has mostly retreated from clinical research. By some estimates, the pharmaceutical and medical-device industries now sponsor about six times as many clinical trials as the NIH. In practice, this dynamic narrows the type of clinical-research questions being investigated; only new drugs and other products that can sell on the market receive rigorous scientific scrutiny.”

There is a hint there that Drs. Cary Gross and Ezekiel Emmanuel don’t differentiate between the NIH’s extramural activities, i.e. awarding grants to entities which are not NIH, its Intramural Research Program which conducts its own — usually high-risk high-reward and sometimes clinical — research, the individual institutes and investigators at the NIH who may contribute to creation of disease guidelines (its work on GVHD assessment is a good example), and the investigators at other institutions who and which may receive NIH funding who also contribute to those guidelines, maybe even as part of an NIH-funded grant.

The rest of the article is as jumbled. It highlights Drs. Kizzmekia Corbett and Barney Graham, correctly naming them “NIH scientists” (they were part of the Intramural Research Program when they worked on what was to become the Moderna vaccine), but as a counterpoint bring up a failing of the extramural program to fund Dr. Katalin Karikó’s mRNA research back in the 1990s. It talks about “NIH-sponsored clinical trials” without qualifying what that means (Which NIH? Financial or IND sponsorship?). It brings up the RECOVERY trial (full disclosure: I am a big fan of the trial) and faults the NIH for… not pushing for a single-payer system and unified electronic medical records that the UK has and which enabled RECOVERY, I guess?

It is clear that NIH needs to rethink its extramural funding mechanisms. Maybe make them more like the intramural program? Make some of them into a lottery (good opportunity for a randomized controlled trial there!)? Fund people not projects, as Gross and Emmanuel (and others long before them) propose?

But several attempts at reform have failed, often because of pushback by incumbent senior scientists who said that whoever proposed the change did not fully grasp what they were dealing with. And you know what: they were right. Do not attempt to reform something which you do not understand. As the first step in understanding, Matt Faherty’s 30+ thousand word report on the NIH is a good start. Mood affiliation pieces like the one in The Atlantic, not so much.