“At a conference a few months before the pandemic, a scholar told me how, in his department, everyone wrote lengthy pre-analysis plans that would, in theory, constrain P-hacking. In practice, he admitted, researchers could give cherry picking free rein, counting on the fact that no one has the time or patience to read a 100-page pre-analysis plan and compare it with the later publication.”

This is from an opinion piece in Nature about reproducibility of science in general, but the sentiment holds for clinical research (arguably, clinical research is even worse, for those hailing results of a clinical trial as game-changing often can’t be bothered with reviewing even a brief ClinicalTrials.gov registration and its history of changes, let alone a 100-page document). You could argue that open analysis plans at least give authors a reason to fear, as a deviation from the plan could result in negative letters to the Editor, retractions, or worse yet, shaming on social networks. In practice, it is those who call out deviations who are more afraid of name calling, Twitter mobs, and employer @-mentions for… being too aggressive, I guess?

Once you incentivize scientists to produce hundreds of pages of text that no human will ever read and no algorithm will parse just so a box or two can be checked, do not be surprised at the outcome.

You don’t have to convince me that NIH grant funding needs to be reformed, but all this article in The Atlantic did was show that even people who get their funding from NIH (i.e. the two authors) have no clue about how NIH operates or what its mission is.

For instance, this assertion:

“… the NIH, the largest public funder of clinical trials in the United States, should also have been well positioned to create treatment guidance for doctors caring for patients hospitalized with a brand-new disease.”

is followed shortly by:

“Moreover, the NIH has mostly retreated from clinical research. By some estimates, the pharmaceutical and medical-device industries now sponsor about six times as many clinical trials as the NIH. In practice, this dynamic narrows the type of clinical-research questions being investigated; only new drugs and other products that can sell on the market receive rigorous scientific scrutiny.”

There is a hint there that Drs. Cary Gross and Ezekiel Emmanuel don’t differentiate between the NIH’s extramural activities, i.e. awarding grants to entities which are not NIH, its Intramural Research Program which conducts its own — usually high-risk high-reward and sometimes clinical — research, the individual institutes and investigators at the NIH who may contribute to creation of disease guidelines (its work on GVHD assessment is a good example), and the investigators at other institutions who and which may receive NIH funding who also contribute to those guidelines, maybe even as part of an NIH-funded grant.

The rest of the article is as jumbled. It highlights Drs. Kizzmekia Corbett and Barney Graham, correctly naming them “NIH scientists” (they were part of the Intramural Research Program when they worked on what was to become the Moderna vaccine), but as a counterpoint bring up a failing of the extramural program to fund Dr. Katalin Karikó’s mRNA research back in the 1990s. It talks about “NIH-sponsored clinical trials” without qualifying what that means (Which NIH? Financial or IND sponsorship?). It brings up the RECOVERY trial (full disclosure: I am a big fan of the trial) and faults the NIH for… not pushing for a single-payer system and unified electronic medical records that the UK has and which enabled RECOVERY, I guess?

It is clear that NIH needs to rethink its extramural funding mechanisms. Maybe make them more like the intramural program? Make some of them into a lottery (good opportunity for a randomized controlled trial there!)? Fund people not projects, as Gross and Emmanuel (and others long before them) propose?

But several attempts at reform have failed, often because of pushback by incumbent senior scientists who said that whoever proposed the change did not fully grasp what they were dealing with. And you know what: they were right. Do not attempt to reform something which you do not understand. As the first step in understanding, Matt Faherty’s 30+ thousand word report on the NIH is a good start. Mood affiliation pieces like the one in The Atlantic, not so much.

“It was a small trial, just 18 rectal cancer patients, every one of whom took the same drug. But the results were astonishing. The cancer vanished in every single patient, undetectable by physical exam, endoscopy, PET scans or M.R.I. scans.”

You shouldn’t judge an article by its headline, so how about the first few paragraphs?

“Dr. Luis A. Diaz Jr. of Memorial Sloan Kettering Cancer Center, an author of a paper published Sunday in the New England Journal of Medicine describing the results, which were sponsored by the drug company GlaxoSmithKline, said he knew of no other study in which a treatment completely obliterated a cancer in every patient. “I believe this is the first time this has happened in the history of cancer,” Dr. Diaz said.”

So far we have “astonishing”, “vanished in every single patient”, “obliterated”, and the “first-time-in-history” gambit. Anything else?

“Dr. Alan P. Venook, a colorectal cancer specialist at the University of California, San Francisco, who was not involved with the study, said he also thought this was a first. A complete remission in every single patient is “unheard-of,” he said.”

“Unheard-of”! So be it.

Not until paragraph 18, long after it praised lack of toxicities in a 12-patient trial of a drug with a known side effect profile, and shortly after mentioning the C-word sandwiched between “remarkable” and “unprecedented” is it revealed that

“The inspiration for the rectal cancer study came from a clinical trial Dr. Diaz led in 2017 that Merck, the drugmaker, funded. It involved 86 people with metastatic cancer that originated in various parts of their bodies. But the cancers all shared a gene mutation that prevented cells from repairing damage to DNA. These mutations occur in 4 percent of all cancer patients.”

The “mutation” in questions is mismatch repair (MMR) deficiency is not actually a mutation, but that is this report’s least egregious journalistic error.

MMR deficient tumors respond well to immune checkpoint inhibitors. One of them, pembrolizumab, has broad FDA approval for all “advanced” (i.e. metastatic or unresectable) cancers with MMR deficiency. The goal of this most excellent study which absolutely should have been done is to see whether it can be used even earlier, to avoid possibly debilitating but potentially curative surgery.

And lo and behold, it can! Congratulations to the study team, immune checkpoint inhibitor manufacturers, and most of all to those people who are yet to develop an MMR-deficient tumor which is still resectable, but now maybe doesn’t need to be resected because it will melt away with immune checkpoint inhibition. NB: this is significantly less than the quoted 4%, which includes people whose cancers are advanced and who can already receive ICIs.

Note that I did not learn any of this from the new article, but rather from being a medical oncologist and reading the NEJM paper which is at least — let it not be said NYT does everything wrong — linked to early on.

How will others read it, one wonders?

“Many breakthroughs in scientific progress have required massive funding and national coordination. This is not one of them. All that needs to be done is allow expert research scientists to do the hands-on work that they’ve been trained to do.”

Spot on.

“We can’t keep making so many scientists tear out their hair over well-intentioned but near-infinite administrative requirements, as if that doesn’t distract from the actual science that they are supposed to be doing.”

Yes!

“…the sheer amount of regulation is so voluminous that if I had to actually read the guidelines that they want us to know about, I would never again be able to submit a grant”.

In communist Russia, the grant writes you!

Silly me, it’s not Russia, it’s the US of A.

He was stage-ready and planning to tape a pair of performances in Los Angeles. Then the coronavirus pandemic hit, shuttering entertainment venues across the nation. At almost the same time, Macdonald’s monthly visit to the hospital revealed that the original cancer, multiple myeloma, had metastasized into myelodysplastic syndrome, which can often lead to acute leukemia. The diagnosis left Macdonald and Hoekstra spinning and unsure of the next steps. Except for one thing: Whatever happened, Macdonald wanted to make sure his material was shown.

There are some great moments in this Washington Post article about Norm Macdonald’s last few years and his last comedy special (available on Netflix starting May 30th), but I couldn’t help noticing a factual error in their description of his medical condition. Macdonald’s multiple myeloma didn’t metastasize to MDS, but was most likely the consequence of prior myeloma treatment. High-dose chemotherapy and autologous stem cell transplantation can cause therapy-related MDS, which is difficult to treat, impossible to cure, and has median overall survival of 18 months.

So why even give this treatment, if a known toxicity is an even deadlier cancer? Because randomized controlled trials showed that people with myeloma who received it lived longer than those who didn’t. And one day, hopefully soon, a randomized trial will show that some other, less toxic treatment is even better than autologous transplantation. Which is to say, it is auto transplant and not randomization that we should strive to send to the dustbin.

“To be able to suppress your dislike or lack of interest in things and just do them to gain an extrinsic rewards like money or grades is a superpower when it comes to being professionally successful, but I… can’t?”

Med school selects for those who can!

Funny that the phrase “stylized facts” started out in economics and is being used in social sciences when 100% of clinical medicine is, in fact, based on stylized facts.

“Natural systems … may look like (a Rube Goldberg machine) superficially because we don’t fully understand what’s going on,” he said. “Once we understand the right way to look at it, we can hopefully appreciate it as a simple design.” Referring to GRNs, but broadly applicable.