Posts in: science

The principal-agent problem of medicine

Alan Jacobs about Amerian health-care, or what passes for it:

I think the first thing to understand about the American health-care system is this: some people lose money from illness, and some people make money from illness. Some people pay, and some people get paid.

I don’t think there are many doctors who consciously make medical decisions based on their lust for money. But I do think there are a great many doctors who go along with the incentives established by the system, without thinking about it too much or at all, because on some level they know that thinking about it could well lead to their losing money.

Of course, most people getting paid from the illness of others are not the doctors, the nurses, or the pharmacists. In fact, outside of lucrative procedure-based specialties — and there aren’t as many of those as a Top Docs glossy would make you think — most doctors, certainly most of those who deal with chronic medical conditions, have no idea how much treatments and tests they order actually cost.

This is, in fact, not a feature but a bug of the system, and one of its main ones. Most doctors work not for their patients, but for amorphous “health systems” graced with all the charm and efficiency of a lumbering bueracracy. They, in turn, deal not with the patients directly, but rather with medical insurance companies or, worse yet, “benefits managers” who insert themselves as mediators nominally there to simplify the process but instead further increasing its complexity. And presto, you now have a series of matryoshka dolls each doing its part to create the mother of all principal-agent problems.

Should the patients' perspective be the primary consideration in improving American health-care? Absolutely! But lets not fool ourselves into thinking that the mess we are in is due to doctors' priorities overwhelming everyone else’s.


Schroedinger's civilization

Niall Ferguson in one of last year’s Conversations with Tyler:

The epistemic problem, as I see it is — Ian Morris wrote this in one of his recent books— which is the scenario? Extinction-level events or the singularity? That seems a tremendously widely divergent set of scenarios to choose from. I sense that — perhaps this is just the historian’s instinct — that each of these scenarios is, in fact, a very low probability indeed, and that we should spend more time thinking about the more likely scenarios that lie between them.

This is bananas thinking! Probability space replacing the river in this well-known Talebism. If the probability space is 4 feet deep on average you don’t just wade into it as if every part is just 4 feet. You need to know the variance, and from Ferguson’s own telling it goes from unlimited upside to complete ruin.

Worse yet: Ferguson is confusing improbable with the impossible. And also hasn’t heard of ergodicity, again courtesy of Taleb. Given a long enough time span, an extremely low-frequency event is a near-certainty. If you don’t believe me, how about a game of Russian roulette?

Is it because Ferguson is a historian? Everything he encounters professionally would have ex post likelihood of 100% so probability theory may not be his area of strength. Don’t ask a historian for predictions, I guess.


Public health, lead time bias, and The Dude

Prof. Devi Sridhar in The Guardian about the epidemic of missed cancer cases:

Early [cancer] diagnosis is important because it improves survival outcomes. In England, more than 90% of people survive bowel, breast and ovarian cancer for at least five years if diagnosed at the earliest stage. This allows treatment to start earlier, before the cancer has spread through the body. Yet even with a cancer diagnosis, the NHS is struggling to provide treatment within the current 62-day target time: 36% of patients waited longer than 62 days in England, 21% in Scotland and 43% in Wales. The main bottleneck is staff shortages, which the Covid-19 pandemic has made more acute. Again, this points to the need for investment in the NHS – in not just infrastructure, but also the workforce.

Prof. Sridhar is chair of global public health at the University of Edinburgh so I was surprised to see her make a basic error in epidemiological reasoning. “Early cancer diagnosis is important because it improves survival outcomes”, the paragraph begins, citing not original research but a comment in The Lancet which, yes, is a prestigious journal, Impact factor 202.731, which is ridiculously high. but calling on comments to back your claims without primary literature are level 0 data.

The Lancet article There is a story here about going down rabbit holes due to poor citation practices — I once spent two days hunting for the primary reference to a single sentence for a letter in a journal nobody reads — that deservs a post of its one. One day., “Earlier diagnosis: the importance of cancer symptoms” does refer to a 2015 systematic review of 209 studies in the British Journal of Cancer — not as prestigious, you’ll notice Impact factor 9. — whose main conclusion was that the studies were of such varying quality that “Heterogeneity precluded definitive findings”.

The authors did speculate in the conclusion that they “believe that it is reasonable to assume that efforts to expedite the diagnosis of symptomatic cancer are likely to have benefits for patients in terms of improved survival, earlier-stage diagnosis and improved quality of life, although these benefits vary between cancers”. Which, fair enough, but: number one, that’s just like, your opinion, man; and number two: there is already a plethora of data about lead time bias fooling you into thinking your early detection prolongs survival when in fact all it did was make the person aware they had cancer for longer without making an iota of difference on when and of what they would die. I base this claim purely on personal anecdote, where people “cured” of their lymphoma were reluctant to get a mammogram — a possible side effect of chest radiation — so they wouldn’t have the aura of cancer hang above them once again. Since this is a situation for which we know that when the cancer does occur, a so-called “secondary malignancy”, it is more aggressive than usual, they ended up doing it, and good for the patient! Yes, there are people who would rather know, but a good proportion — as this is a blog post and not a commentary in The Lancet I am going to allow myself some speculation here — possible more than half would rather not!

So what is going on here? Surely the chair of global public health at a well-known university knows about the lead time bias? The last three years made me question jumping to that conclusion right away, but let’s give some benefit of doubt. The key word here, I’m worried, is public health, a blunt-force instrument which does away with nuance in favor of broad if not deep messages and interventions. Sometimes these are terrifyingly successful: witness the eradication or near-eradication of infectious diseases, or my favorite — plummeting smoking rates in the United States after a public campaign and a flurry of lawsuits that saved more lives than all statins and chemotherapeutics put together. But the dangers of oversimplification are real, like the crusade to ban saturated fats in favor of simple processed sugars backfiring spectacularly. Caveat audiens.

So anyway, that’s why I don’t read newspaper coverage of medical matters, opinion pieces, or The Guardian.

Thats just like, your opinion, man

In the late 1700s, America’s founding fathers had a fight over the best treatment for yellow fever, then sweeping though Philadelphia. Republicans preferred bloodletting and purges; Federalists took a more conservative approach.

If it’s any consolation, the country survived.


Do better cancer drugs cost more?

No.

To quote myself and colleagues in JAMA Internal Medicine:

Among the drugs approved on the basis of response rate, there was only a weak correlation between cost and the magnitude of the response rate gain; the same was true in the categories of drugs approved on the basis of PFS and OS gains. This suggests that cancer drugs are priced based predominantly on what the market will bear. Correcting this trend is vital for the solvency of health care and pharmaceutical development.

The following caveats apply:

  • We looked at median wholesale price, not the retail price. Wholesale cost is what the drug companies think their products are worth. Retail price is what the markets think they are worth. But there are such things as anchoring and, as the accompanying editorial nicely reviewed, the biggest players on that market Medicare and Medicaid are forbidden by law to bargain.
  • We only reported overall survival (OS), progression free survival (PFS) and overall response rate (ORR). We looked at other things as well, like mechanism of action and how novel the drug is, which will be reported separately.
  • We only looked at the last 6 years of approvals.
  • We only looked at cancer drugs.

But if you are a medical student or a resident and want to do some important work while buffing up your CV, feel free to copy our methods and apply them to other areas of medicine! Something like this for cardiology or endocrinology would do a whole lot more good than yet another case report of something presenting as something else.


How Unforced Errors Hobbled America's Monkeypox Response

Katherine Eban at Vanity Fair:

Though Fenton is a FEMA superstar with ample experience responding to tornadoes and hurricanes, it would have been more logical for the top person to come from within the HHS family of agencies, though a division director from the Centers for Disease Control and Prevention (CDC) was assigned as Fenton’s deputy. The choice “reflects the fact that CDC can’t operate its way out of a paper bag,” said the former HHS official.

Third year into the covid-19 pandemic and I still can’t wrap my head around the fact that the CDC is moribund. Institutional decay comes for all during the fat and lazy times.

Katherine Eban also wrote this brilliant account of the lab-leak hypothesis and a hair-raising book about the FDA that made the agency’s fumbles, unlike the CDC’s, not at all surprising.


Donald McNeil, formerly of the New York Times, wrote a primer on the monkeypox outbreak which is well worth the 10 minutes' reading time. The bottom line: not great, not terrible. For now.


“This was an ambitious report recommending all sorts of ways to reform government, but no one was given a mandate and timeline to actually carry out the recommendations.”

Thus ends every attempt to reform administrative burden of research, according to the Good Science Project.


Social sciences aren't the only ones with reproducibility issues

The Reproducibility Project: Cancer Biology was an 8-year effort to replicate experiments from high-impact cancer biology papers published between 2010 and 2012.”

Out of 193 experiments from 53 papers, only 50 (26%) were successfully reproduced, and in those the effect sizes were 85% smaller on average. Scientists at Bayer did the same thing 10 years ago, with identical results: only 20-25% of experiments reproduced.

With foundations like this, it is amazing that there has been any progress in the clinic.


Several imprecisions in this essay on IRBs should not detract from its key point: social sciences don’t need IRB oversight, biomedicine needs it to be less byzantine and more transparent. Status quo is untenable.