There was something particularly irksome about a USA Today article from a few days ago — it prompted 3, count them, three tweets posts Xs from me — and I wanted to figure out what bothered me so. Here is the headline:

Left or right arm: Choosing where to get vaccinated matters, study suggests. Here’s why

No, it’s not the typography, although they should either not have had a full sentence in their headline, or else should have finished it with a full stop. But then they would have lost the chance for the click-baity Here’s why as a prelude to an article ⊕ OK, this can get real confusing real fast since there are two articles I am writing about: the USA Today’s newspaper article, and the research article to which it refers. So, let’s use article for the newspaper, and manuscript for the research article. Because why not? about Real Science™ which — color me astonished — takes a hypothesis-generating study and presents the hypotheses it generated as the final results.

To its credit, the article starts of with a link to the manuscript and the name of the journal where it was published, which is eBioMedicine, part of the proliferating Lancet family, impact factor 11.1. ⊕ Although, you know what they say about impact factors.Good! They also invited an independent researcher to comment. And I am sure that his comments were similar to mine, although of course most of what he said (or more likely wrote in an email) didn’t make it. What ended up on the page were two blurbs about precise vaccination from the director of a Precision Vaccines program. Gasp.

But these are all side attractions. The biggest problem is this: scientists want to compare people who had a two-dose vaccine shot in the same arm to those who had it in different arms; in the manuscript, these were called ipsilateral and contralateral groups. They aren’t randomizing people to one versus the other, ⊕ What they describe as randomization isn’t really so, but that’s a rabbit hole we better not get into. but with these being generally healthy people, and with the participants not having a choice as to where they will get a vaccine, that is not too much of an issue. Then they ask them some questions about vaccine side effects and draw some blood. The questions are about side effects and the blood is to check for “the strength of the immune response”.

Note that they don’t say at the outset that the groups would be different, and how. Would the opposite arm have fewer side effects? Better immune response? If so, in what way? More antibody? Stronger antibody? A different subtype of antibody? Better or worse cellular immunity? Which cell (among dozens)? More cells, stronger cells, or different cells? Or maybe the same side would be better?

The beauty of hypothesis-generating research (for the researcher) is that it doesn’t matter. Whatever you get, you will get it published, sometimes in a double-digit impact factor publication. I’ve sat on many a lab meeting where things like this were proposed and always, always, the comment is that “the results will be interesting whatever they are”. And they are right! But you will not know — cannot know — whether the results you got are based on an underlying physiology, or occurred purely by chance. That is where confirmatory studies come in.

Neither the manuscript nor the article recognize this. Among the many things they looked at, the researchers found two things that were different between the two groups: those who had the vaccine in the same arm had “more” of a certain type of immune cell than the other, and the opposite-arm group had increased expression of a certain marker on yet another type of immune cell. “More” is in quotes because even that is more subjective than it appears — another rabbit hole — but even if true in this sample, it is at best a hypothesis that should lead to another, possibly smaller study, where you focus on these cells, with different operators counting them, and doing additional hypothesis-generating analyses on the side to figure out the why of it, which would lead into yet another confirmatory study… You get the idea.

This is not what the manuscript authors propose. Instead they take their result at face value and concoct a mechanism out of thin air that would explain the result. The journalist then takes the mechanism and presents it as the main research result, the Here’s why of that clickbait headline. There is a high bar for calling anything in science conclusive and the article does have the usual disclaimer that “more research and data is needed”. But the phrase has been repeated so much that it has lost all meaning, something you say to mark yourself as a “believer in science” while with a wink and a nudge you act as if the results were indisputable.

Fortunately, science is a strong-link problem: those who know what they are doing will adjust their beliefs accordingly, and down the line confirm or falsify these preliminary findings. Unfortunately, science doesn’t operate in a vacuum. If its covering of science is indicative, journalism, the fourth estate, is in a hole and digging deeper, taking others with them.

• As of this morning it has reached >17,000 signatures. Yay!
• The Healthcare Unfiltered podcast has two good interviews: the first is with Wes Fisher and his decades-long battle with ABIM; the second is a debate between Aaron Goodman — who started the petition — and Richard Baron, the President and CEO of ABIM
• The debate was a clear loss for Baron: his smooth baritone and even smoother sophistry could not help hide the fact that MOC is a pure money grab. I will lay out the 3rd grade math and 6th grade logic later this week.
• Aaron was too nice on Baron. There should be a second petition: for ABIM to start having accountability, or else shut down. Back in the 1930s when it was founded we may have been ABIM “diplomats” who pay for their certification once then never again. Once you start extracting yearly dues, you no longer have gentlemen diplomats, you have members.
• Richard Robber Baron.

Nassim Taleb has updated his essay against IQ, and I don’t know if Figure 1 there is new or I haven’t been paying attention before, but it is a true eye-opener. It shows how meaningless correlation is in the absence of symmetry, and medicine is full of asymmetries. I shudder to think how much medical literature consists entirely of physicians-cum-naïve statisticians pouring through medical charts gathering data to calculate such correlations. Counting the official and semi-official guidelines based on such flawed papers would be a nice side project.

Aaron Goodman’s petition to eliminate MOC requirements has 16,000 signatures and counting. That’s a lot, but still <10% of ABIM’s 220,000+ active certificate holders for internal medicine alone. So, please sign if you haven’t already!

And if you have no idea what any of these acronyms mean, ABIM’s home newspaper of record has a good overview.

“The American Board of Internal Medicine is under fire for a ‘maintenance of certification’ requirement. Their own tweet didn’t help.":

Frustration among physicians who feel they are being asked to do increasingly more to prove their competency has been building for years and in recent weeks, boiled over for many. At least 12,000 people have signed a Change.org petition, which is open to anyone. Many added their name after the petition’s organizer resurfaced a July tweet in which ABIM suggested their ongoing certification was so easy, doctors could do it while on vacation.

I have been meaning to write about ABIM’s train wreck ever since I signed the petition, but yet again my proscratination has been awarded: Philadelphia Inquirer says everything I wanted to, and then some. Ding-dong…

Economists do occasionally publish papers with which I agree: ⊕ For the other ones, look here.

…I find that vaccines saved 748,600 lives through June 2023. That is, without vaccines, cumulative mortality from COVID-19 would have been closer to 1.91 million over this time period. In answering the second question, I find that behavioral efforts to slow the transmission of the virus before vaccines became widely administered were critical to this positive impact of vaccines on cumulative mortality. For example, with a complete relaxation of these mitigation efforts, vaccines would have come too late to have saved a significant number of lives. Earlier deployment of vaccines would have saved many lives.

Which yet again shows that out of the two extremes, John Snow and GBD, Snow was the more correct one both ex ante and ex post. Yet instead of taking an “L”, GBD proponents keep saying that we should not have locked down. Yes, shutting down outdoor playgrounds for a full year was ridiculous, but stopping mass gatherings and any goings on in tight public spaces until we get a working vaccine? Absolutely! Only next time, Operation Warp Speed should be set at Warp 5, not 0.1.(ᔥTyler Cowen)

The Washington Post’s Andrew Van Dam on the average US doctors' salaries:

The average U.S. physician earns $350,000 a year. Top doctors pull in 10 times that.

I will write more about this later but for now I will just note how frustrating it is to read an article that has a premise and conclusion that I completely agree with (America doesn’t have enough doctors so the ones that it does have are compensated way above average) backed up by mishandled and misreported data (first the article doesn’t say whether the “average” is mean or median — it is the median, which is actually good — then doesn’t explicitly mention that the median in question is of the adjusted gross income at the household level, not of individual compensation: the median total individual income is $265,000).

At least the article linked to the NBER paper with all the data, which in turn completely validated my recent quip about economisits. Frustrating throughout, especially if you try reading the comments.

Economists are prone to making hypotheses about other fields that make perfect sense to them and others outside of the field, but that can be easily refuted to anyone with an iota of relevant field-specific knowledge. And not just economists. This very sentence is, in fact, one such hypothesis.

But to get to the point: Alex Tabarrok at Marginal Revolution wrote a post titled Left Digit Bias in Medicine which excerpted his WSJ review of Random Acts of Medicine which is a book about “The Hidden Forces That Sway Doctors, Impact Patients, and Shape Our Health”, which is, believe it or not, part of the title. ⊕ I hate, hate, hate what book titles have become. This is the literary equivalent of the Amazon product name trash recently discussed on ATP. SEO for books as an externality of Amazon’s dominance — who would have guessed?

Now, I haven’t read the book yet, and the WSJ is behind a paywall, so I only have the excerpt to go by, but it is long and it is sufficient. I won’t quote from it — 4th level of abstraction would be too much — but I will copy the figure and make a few comments. You should read the blog post itself, it is good.

1. This is the biggest and the most obvious regression discontinuity I have seen, and it has a reasonable explanation. Kudos.
2. Age is plotted as a continuous variable. This is not how doctors see the patient’s age in their medical record. It is shown as an integer, not a fraction, so someone who is a day shy of their 40th birthday will look just the same as someone who just turned 39. And if the guidelines say you should do something for a 40-year-old but not for a 39-year-old, that’s what most doctors — let’s hope — will do.
3. This is therefore not left-digit bias.
4. While the date of birth is also part of the medical record, it is rarely if ever looked at by MDs — except just prior to an invasive procedure that requires a timeout. It is often checked by nurses prior to administering medications, and they are often the ones who will note that an inpatient’s birthday is coming up.
5. Tabarrok has buried the lead in his blog post. Regardless of the cause, the discontinuity is there and can be used as pseudo-rendomization for a natural experiment of the effect of “testing for heart attacks” (I will guess by that the others meant an ECG and troponin levels) on outcomes. The entire last paragraph of the excerpt is about that, and I 100% endorse the idea.

Two things keep me logging back in to Twitter X: DMs from people who should know better, and all of my colleagues who insist using it. But how else was I supposed to learn of this petition for ABIM to eliminate their “maintenance of certification” grift?

Yesterday’s EconTalk was with Lydia Dugdale on the Lost Art of Dying, which is the title of Dr. Dugdale’s book but also a translation of Ars moriendi, a 15th century Latin text about the good death. The episode is in this year’s Top 5, and I wish I could dwell into this. Ars longa…, as they say.