Public health, lead time bias, and The Dude
Prof. Devi Sridhar in The Guardian about the epidemic of missed cancer cases:
Early [cancer] diagnosis is important because it improves survival outcomes. In England, more than 90% of people survive bowel, breast and ovarian cancer for at least five years if diagnosed at the earliest stage. This allows treatment to start earlier, before the cancer has spread through the body. Yet even with a cancer diagnosis, the NHS is struggling to provide treatment within the current 62-day target time: 36% of patients waited longer than 62 days in England, 21% in Scotland and 43% in Wales. The main bottleneck is staff shortages, which the Covid-19 pandemic has made more acute. Again, this points to the need for investment in the NHS – in not just infrastructure, but also the workforce.
Prof. Sridhar is chair of global public health at the University of Edinburgh so I was surprised to see her make a basic error in epidemiological reasoning. “Early cancer diagnosis is important because it improves survival outcomes”, the paragraph begins, citing not original research but a comment in The Lancet which, yes, is a prestigious journal, Impact factor 202.731, which is ridiculously high. but calling on comments to back your claims without primary literature are level 0 data.
The Lancet article There is a story here about going down rabbit holes due to poor citation practices — I once spent two days hunting for the primary reference to a single sentence for a letter in a journal nobody reads — that deservs a post of its one. One day., “Earlier diagnosis: the importance of cancer symptoms” does refer to a 2015 systematic review of 209 studies in the British Journal of Cancer — not as prestigious, you’ll notice Impact factor 9. — whose main conclusion was that the studies were of such varying quality that “Heterogeneity precluded definitive findings”.
The authors did speculate in the conclusion that they “believe that it is reasonable to assume that efforts to expedite the diagnosis of symptomatic cancer are likely to have benefits for patients in terms of improved survival, earlier-stage diagnosis and improved quality of life, although these benefits vary between cancers”. Which, fair enough, but: number one, that’s just like, your opinion, man; and number two: there is already a plethora of data about lead time bias fooling you into thinking your early detection prolongs survival when in fact all it did was make the person aware they had cancer for longer without making an iota of difference on when and of what they would die. I base this claim purely on personal anecdote, where people “cured” of their lymphoma were reluctant to get a mammogram — a possible side effect of chest radiation — so they wouldn’t have the aura of cancer hang above them once again. Since this is a situation for which we know that when the cancer does occur, a so-called “secondary malignancy”, it is more aggressive than usual, they ended up doing it, and good for the patient! Yes, there are people who would rather know, but a good proportion — as this is a blog post and not a commentary in The Lancet I am going to allow myself some speculation here — possible more than half would rather not!
So what is going on here? Surely the chair of global public health at a well-known university knows about the lead time bias? The last three years made me question jumping to that conclusion right away, but let’s give some benefit of doubt. The key word here, I’m worried, is public health, a blunt-force instrument which does away with nuance in favor of broad if not deep messages and interventions. Sometimes these are terrifyingly successful: witness the eradication or near-eradication of infectious diseases, or my favorite — plummeting smoking rates in the United States after a public campaign and a flurry of lawsuits that saved more lives than all statins and chemotherapeutics put together. But the dangers of oversimplification are real, like the crusade to ban saturated fats in favor of simple processed sugars backfiring spectacularly. Caveat audiens.
So anyway, that’s why I don’t read newspaper coverage of medical matters, opinion pieces, or The Guardian.
In the late 1700s, America’s founding fathers had a fight over the best treatment for yellow fever, then sweeping though Philadelphia. Republicans preferred bloodletting and purges; Federalists took a more conservative approach.
If it’s any consolation, the country survived.
Do better cancer drugs cost more?
No.
To quote myself and colleagues in JAMA Internal Medicine:
Among the drugs approved on the basis of response rate, there was only a weak correlation between cost and the magnitude of the response rate gain; the same was true in the categories of drugs approved on the basis of PFS and OS gains. This suggests that cancer drugs are priced based predominantly on what the market will bear. Correcting this trend is vital for the solvency of health care and pharmaceutical development.
The following caveats apply:
- We looked at median wholesale price, not the retail price. Wholesale cost is what the drug companies think their products are worth. Retail price is what the markets think they are worth. But there are such things as anchoring and, as the accompanying editorial nicely reviewed, the biggest players on that market Medicare and Medicaid are forbidden by law to bargain.
- We only reported overall survival (OS), progression free survival (PFS) and overall response rate (ORR). We looked at other things as well, like mechanism of action and how novel the drug is, which will be reported separately.
- We only looked at the last 6 years of approvals.
- We only looked at cancer drugs.
But if you are a medical student or a resident and want to do some important work while buffing up your CV, feel free to copy our methods and apply them to other areas of medicine! Something like this for cardiology or endocrinology would do a whole lot more good than yet another case report of something presenting as something else.
How Unforced Errors Hobbled America's Monkeypox Response
Katherine Eban at Vanity Fair:
Though Fenton is a FEMA superstar with ample experience responding to tornadoes and hurricanes, it would have been more logical for the top person to come from within the HHS family of agencies, though a division director from the Centers for Disease Control and Prevention (CDC) was assigned as Fenton’s deputy. The choice “reflects the fact that CDC can’t operate its way out of a paper bag,” said the former HHS official.
Third year into the covid-19 pandemic and I still can’t wrap my head around the fact that the CDC is moribund. Institutional decay comes for all during the fat and lazy times.
Katherine Eban also wrote this brilliant account of the lab-leak hypothesis and a hair-raising book about the FDA that made the agency’s fumbles, unlike the CDC’s, not at all surprising.
Donald McNeil, formerly of the New York Times, wrote a primer on the monkeypox outbreak which is well worth the 10 minutes' reading time. The bottom line: not great, not terrible. For now.
“This was an ambitious report recommending all sorts of ways to reform government, but no one was given a mandate and timeline to actually carry out the recommendations.”
Thus ends every attempt to reform administrative burden of research, according to the Good Science Project.
Social sciences aren't the only ones with reproducibility issues
“The Reproducibility Project: Cancer Biology was an 8-year effort to replicate experiments from high-impact cancer biology papers published between 2010 and 2012.”
Out of 193 experiments from 53 papers, only 50 (26%) were successfully reproduced, and in those the effect sizes were 85% smaller on average. Scientists at Bayer did the same thing 10 years ago, with identical results: only 20-25% of experiments reproduced.
With foundations like this, it is amazing that there has been any progress in the clinic.
Several imprecisions in this essay on IRBs should not detract from its key point: social sciences don’t need IRB oversight, biomedicine needs it to be less byzantine and more transparent. Status quo is untenable.
Give a lecture once and you help a few hundred people (if you’re lucky). Post the lecture to YouTube and you help millions. 7 years later, I am yet to find a better guide to academic writing.
NB: it’s good to have a live audience.
“Whether hot or cold water freezes faster remains unknown.”
Thus begins a wonderful Quanta Magazine article about the “Mpemba effect”, named after a Tanzanian teenager who saw something funny happen to his home-made ice cream. Reality is complicated.