Posts in: medicine

News distortion, a case study

The headline: ChatGPT appears to pass medical school exams, educators rethinking assessments.

The article:

  • They were mock, abbreviated exams,
  • done incorrectly, There are no open-ended questions on the real USMLE.
  • which it didn’t actually pass,
  • and which were reported in a pre-print. Which isn’t a complete knock against the study per se, but even a glance at it shows that some questionable choices have been made regarding the scope — there were only 376 publicly available questions instead of more than a 1,000 on the real exams — and the methods used to ensure the publicly available questions hadn’t already been indexed by the ChatGPT training algorithm.

To be clear: this is my complaining about misleading headlines, not saying that predictive AI wouldn’t at some point be able to ace the USMLE, that point not being now, for reasons stated above. And let’s not even get into whether having a high USMLE score means anything other than the person achieving a high score being a good test-taker (it doesn’t).

A yearly theme, of sorts

Instead of setting a Yearly Theme A CGP Gray video is where I first heard it used as a replacement for New Year’s resolutions, but I’m not entirely sure if he’s the originator. right at the outset, I let it crystalize on its own in the first few months of the year. The theme of 2022 was shelter-building — guess where that came from — and as a result we now have a whistle-clean basement ready to serve as a home gym until a nuclear strike anhilates us all.

Odds are that this year’s theme will end up being statistical shenanigans. First a brief letter to JAMA Internal Medicine we wrote received a confused commentary from a giant of cancer care that showed that even oncology giants are not immune to errors Finding the error I will leave as an excercise for the reader; I do, however, plan to address it in a follow-up letter. Never pass an opportunity to increase your publication count! of statistical reasoning. Soon after that, working on a different — still top-secret — paper got me down a rabbit hole of the many ways we use to present clinical data. I thought these were lacking in oncology; other fields of medicine showed me that there was room for further deterioration. Not to be so secretive about everything, but clinical data representation in this particular field will also be the subject of a commentary. And yet, the US FDA still thinks statistically illeterate doctors — present company included — are important gatekeepers of diagnostic tests, essentially banning home test kits available in other parts of the world because they are worried people are too innumerate to correctly interpret their own results.

Humans being pattern-recognition machines, I don’t doubt I will continue seeing matemathical malpractice, malfeasance, and just plain stupidity everywhere I look. It is pretty much guaranteed I will inadvertently comit some myself! I hope this yearly theme results in a few papers, at least.

From the Annals of Internal Medicine: Curiosity

Old (1999), but still good. Even when I first wrote this, and even older now.

When I was a house officer and installing one of the first right-heart catheters, the machine that showed intrapulmonic arterial pressures was enormous and was equipped with strain gauges rather than computer chips. Making it work was difficult. After the line was in, the attending, the nurse, and I tried desperately to adjust the machine to show the pulmonary arterial pressure waves. We could not get them. The line on the screen remained flat. We manipulated toggle switches and strain gauges for about 15 minutes. Nothing. Finally, I glanced at the patient: He was dead.

The story after that is even better.

A short list of earnest but misguided attempts to reduce costs in medicine

  • Fractional use of vials/pills to decrease per-patient cost, because the main driver of high cost of drugs is not manufacturing (i.e. a ten times more efficient manufacturing process would not result in 10 times, or even 2 times lower prices). If you don’t believe me just look at what Sanofi did with alemtuzumab.
  • Using real-world data instead of randomized controlled trials, because while retrospective, non-randomized, uncontrolled studies Now rebranded as “real-world data”. are good for generating hypotheses and maybe, maybe, detecting enormous effect sizes Think: smoking causing cancer, but not: who-knows-what new material causing lymphoma. we have learned through much trial and error that RCTs are critical for evaluating whether a medical intervention works or not. Back when personal computers were too big and expensive for mass use, the answer wasn’t to invent a story of why calculators were better — it was to make PCs so cheap and small that not having one in your pocket was a matter of personal choice, not cost. Same for RCTs.
  • The Choosing Wisely initiative, which was all the rage back when I was a resident and still seems to have legs. Not to mention that the program unintentionally promoted a dangerous frame of mind in which some doctors thought extensive testing was never indicated, thus missing some rare but life-theratening diagnoses. Money spent on producing more content for doctors to read, listen, and watch — thus taking up their time — and encouraging patients to talk to their doctors at length about questionable data behind many of the procedures — thus, again, taking up their time — may have been better spent designing and running pragmatic RCTs that would answer these questions and save both the doctors' and patients' time by reducing ambiguity. Oh well.
  • Yet another health care delivery reform — this may be just my healthcare policy naiveté, but these all had the whiff of rearranging deck chairs on the Titanic. See also: the Homer Simpson car Mandating the desired outcome instead of thinking about the right incentives is bound to increase cost through second-order effects.

Anything else?

Corona 300

March 7, 2020 was a Saturday. I woke up at 8am, which is as late as it gets, since the night before we watched Breathless and The Graduate back-to-back (the 1960s were a good decade for movies). Most of they was spent in visiting friends in downtown DC. They are a family of four in a tiny one-bedroom; we compared notes on where best to stash the extra flour, rice, pasta, and other staples Though not, funnily enough, toilet paper. we stocked up on expecting the inevitable. The inevitable came that night as we were heading out, when Mayor Bowser announced in a late news conference that yes indeed Washington DC had its first confirmed case of Covid-19: a man with no recent travel and no confirmed exposures, which is to say, there was already community spread. We got back to our apartment and closed the door; the next time that apartment would be empty of people again, as it usually had been on weekends and later summer afternoons before the pandemic, was more than five months later.

That was 300 days ago to the day, and as my favorite columnist and fellow millennial Janan Ganesh astutely noted, there were no grand lessons that these 300 days gave me, unless you count confirmation that humans can muddle their way through anything as a lesson. Harambe may have been killed in 2016, but 2020 was his year: a tragic, sensless event where everyone is responsible but no one is to blame — though I may be an exception in thinking this, since 2020 was the year of confirmation bias, the year of suppressing the opposing view points, the year of shaming. To complicate matters some more, it was also the year when crackpots and idiots joined into the Grand Coalition of Stoopid, expressing some points of view that maybe ought to be suppressed, and doing some things for which maybe they should be ashamed. Harambe indeed.

I finished the last year with a post about the great things that happened to me personally as the world stagnated in the 2010s. In the spirit of this year, I’ll finish with a list of failures instead, and I’ll do my best not to make it into a thinly veiled list of successes:

  • I read far fewer books and watched far fewer movies than any year before.
  • I wrote far fewer (medical) articles than planned.
  • I wasted time on Twitter like never before (and, let’s hope, never again).
  • I dropped more projects than ever before, including piano lessons, learning a new language, speed-completing the Rubik’s cube, and running in cold weather, among many others.
  • I walked less than any other year since I started walking. This may be an exaggeration, but not by much.
  • I commuted more by car than ever since moving to DC.
  • I ordered more takeout than ever.
  • And the one that hurts the most: I did not finish a single video game, or even play anything for more than 15 minutes, unless you count Good Sudoku which is truly a masterpiece of design and the highlight of the year. Yes, the highlight.

Blogroll

I, for one, am glad that blogs are making a comeback. Here are a few I’ve been reading for at least a few months, many of them for years, some for decades.

Applied philosophers

The only true philosophers of our time.

The new scientists

People without major academic credentials who have interesting ideas about science.

The old scientists

People with major academic credentials and interesting ideas, something to teach, or both.

The ludites

People against modernity of one sort or another.

People doing their own thing

Unclassifiable but exhilarating.

Apple enthusiasts

Some tips, a few tricks, many opinions.

Finance-adjacent

Economists and investors, for the most part.

Journalist-cum-substackers

Former or current journalists who now earn some or all of their living by writing newsletters via Substack, which is slowly reinventing blogs (in the sense of reinventing the wheel, not actually making them better and in fact in many was making them much worse).

Company blogs

For when I really want to know when the next update is coming.

Some observations on Covid-19 from recent personal experience

  • A few hours before I developed symptoms I had a negative screening nasal swab. By the time I got a positive test three days later the symptoms were well on their way to resolving. Good thing I didn’t believe that first result.
  • What helped my not believing was that I had textbook Covid-19 which was moderate bordering on severe: fever 39.5°C (~103°F), chills, body aches, nasal congestion, rhinorhea, and a dry cough that was mild enough for me not to worry. But thankfully no anosmia.
  • Read the preceding paragraph again. The nasal swab done just before I developed all those symptoms (and arguably while having chills - though I didn’t know they were chills at the time) was negative. Covid-19 testing is no better or worse than any other clinical test we have, which is to say caveat medicus.
  • Considering our family’s practices I was surprised that it managed to get in and suspected it was one of the new strains. Lo and behold not 7 days later the UK strain was found in Maryland. I won’t know the sequence of the one that got me for a few more months, but I’d say it’s likely.
  • Said practices did contribute to containment, as there seemed to be no spread outside of the household (there is a small asterisk there which I will leave for another time).
  • The new strains being so much easier to get makes any delays in administering the vaccine that more deadly. This is hard to overstate: shots in arms now, doesn’t matter how and to whom.
  • Speaking of shots, I did get my first dose a few days before the likely exposure, and plan on getting the second one as scheduled if available.
  • Masks aren’t 100% effective, particularly in areas of high prevalence which is right now most of the world. The new strains shift the equilibrium even more. Holier-that-though memes about things being OK again if only people did what’s good for them (i.e. wore a mask) are misguided at best and quite likely counterproductive.
  • Another misguided effort: a DC health professional telling the sole member of a large family without a fever to use a separate bathroom, wear a mask at all times and open all the (quite tall) windows of their 1200 sq ft 7th floor apartment. Hard to tell if this was more comical or dangerous.
  • DC health professional’s misguided advice #2: to get everyone in the household tested. If mine was positive and four more people also have fevers do we really think they have something else? Why risk the tester’s exposure and waste reagents: count these people as positive and move on.
  • But as things stand right now, if these household members don’t get tested they don’t count as positive. How prevalent is this situation, I wonder? Even with test availability not being a bottleneck I’d multiply the current counts by at least 2, probably 3 to get the real number (and I’m sure there are epidemiologists who have a more scientific explanation for why we should be doing that anyway).
  • Symptoms in children seem to be no different than any other febrile viremia of childhood (and in fact may be slightly better as they didn’t seem to sap any of their energy, for better or worse). Does this make in-person school more or less safe? I can see both sides of the argument but if you thought children as asymptomatic carriers would be a big risk that risk is probably overblown as they do in fact get symptoms — they just won’t telegraph them.
  • And if you are worried about long-term effects of Covid-19 in children, well, sure, but how is that different from long-term effects of any febrile viremia of childhood? I’m sure our parenting style will ruin their prospects enough that Covid-19 will be just a drop in the bucket.
  • I have been getting lists of home remedies from people who should know better. This includes aspirin (as an anti-platelet agent, not an antipyretic), azithromycin (still!) zinc, turmeric, propolis. What I took: a little bit of APAP and a lot of H2O.
  • I have a new appreciation for the gig workers, who are the unsung heroes of the pandemic. Tip your Dasher.
  • 2021 is certainly off to an interesting start.

It's time to stop the foreign doctor kabuki

Residency application season has just started. Many of the applicants, a few of whom I know in person, will be foreign medical graduates, or FMGs, meaning that they are doctors who want to work in the US but are not US citizens. Most FMGs, but not all, will also be international medical graduates — IMGs — meaning that they have graduated from a non-US medical schools. Something called the Education Commision for Foreign Medical Graduates, or ECFMG, acts as their medical school when interacting with most of the sprawling US bureaucracy. These are our personae dramatis, if you will.

Disclosure: I am both an FMG and an IMG, and first began working in the US on an ECFMG-sponsored J1 visa.

America is a net importer of physicians, that much should obvious to anyone who’s ever been in an American hospital. The country depends on FMGs to keep the system running, get the less lucrative specialties, work in underserved areas, etc. Not so obvious is that most FMGs get to America by lying; ICE-approved, foreign-government sponsored lying for sure, but lying nonetheless.

Here are the lies FMGs tell when they come in: that their country has a need for doctors of such-and-such specialty, and/or that their government is sending them to the US for training in the said specialty, and/or that at the end of training they will go back to their country of origin to work in the (sub)specialty they came in to obtain. Those are the three postulates of the J1 physician exchange visa, the very name of which is also a lie as there is no exchange taking place: foreign doctors do come in, but no American doctors come out.

The postulates are incompatible with reality, and imply foreign government competence that just isn’t there in second and third-world countries. The transitioning and developing world, if you will. Because over there, no one is keeping statistics on specialist needs, and if they are there is actually a surplus, and if there isn’t they wouldn’t be able to afford the (sub)specialists once they come back, and if they could then they would be chosen by party or family lines, and you wouldn’t want them in your hospitals anyway.

So to get a J1 visa FMGs need to obtain a letter from their Ministry of Health or equivalent stating the above (the postulates, not the actual truth; I’m sure that in some of those countries people have gone to prison for saying the truth). But is there a functioning Ministry of Health? Does anyone there know that the letter they are supposed to provide about lending a medical graduate and wanting them back is a piece of kabuki theater, and not a commitment to employ that person if and when they come back? And because this letter is supposed to come in a sealed envelope directly from the Ministry to ECFMG: does anyone there speak English? So here are all those FMGs whose main reason to emigrate to America may have been to escape their kleptocratic governments, being dragged into a game of Whom do I bribe next? and Which newspaper do I threaten them with? In 2019 the correct answer is, for most countries of this sort, None. by the rules of the country they were hoping was less crooked than their own.

Which is fine for America, because it doesn’t care as long as it gets its steady stream of MDs one way or another. Only it should care because 1) the amount of person-hours wasted is on par with if not greater than the amount spent writing grants, and that one’s a whopper, 2) it relinquishes control over a part of its healthcare to foreign governments, and 3) it introduces an air of subterfuge and deceit at the very beginning of the FMG-USA relationship. I would like to think this is an aberration to be fixed, and not a preview of things to come in other areas of governance.

The process was probably fine 50 years ago, when both demands of the medical system and the influx of foreign doctors were but a fraction of the current monstrosity, when USMLE was taken on paper if you had to take it at all, when it wasn’t so obvious to a non-aligned physician whether they should go to the US or USSR (or Yugoslavia, for that matter) to get more training. But healthcare has changed and so has the world: it’s time do drop the pretense of an exchange, America, and be honest about what’s going on here.

What I believe that most people probably don’t (no data behind this, just the armchair)

The world in general, and the US in particular, is spending too much on goal-directed, targeted biomedical research while undervaluing both applied and theoretical physics. Picture Leonardo da Vinci drawing helicopters: that’s the modern-day cancer researcher. The universal cure for cancer — and there should be one, if humanity survives long enough to create it — will not come from an NIH grant. If grants are involved at all, it will be something initially funded by the National Science Foundation. The current system of funding (government, non-profit, biotech, you name it) is broken, and if you account for the opportunity cost it is a complete disaster. Each of these statements deserves at least a paragraph, but I am saving my carpal tunnels for a manuscript, an LOI, and a couple of protocols (oh, the irony).

In the meantime, a few things physician-scientists should do for the overall good:

  • find causes and create better prevention strategies, because a look at the SEER database will tell you that it’s not just bad luck;
  • eliminate barriers for administration of known curative therapies world-wide (do we really want to leave this to politicians and economists?);
  • ensure rapid and honest evaluation of the many new treatments, procedures, and diagnostic/prognostic methods coming out of the biomedical behemoth.

How beneficial any of this would be for one’s career is a different question altogether, but let’s not get into incentives because RSI. I am also very open to opposing opinions, since my being wrong would make my life easier.

Level up

The next time someone asks me about books to read before residency, I will direct them here. You don’t have to be a medical trainee to benefit from these, but that period of anxious anticipation between match day and orientation is perfect for buffing your attributes.

How to read a book, by Mortimer J. Adler

What better way to start learning about learning than by reading a book about reading books?

The Farnam Street blog has a nice outline of the book’s main ideas. The same establishment is now hocking a $200 course on the same topic. It’s probably good, but at $10 the source material is slightly more affordable.

Getting things done, by David Allen

The first few months you will be neck-deep in scut work no matter what you do. After that, though, you will have to juggle patient care, research, didactics, fellowship/career planning, and piles of administrative drek—and that’s just inside the hospital. At the very least, this book will help you make time for laundry (and maybe some reading).

Thinking, fast and slow, by Daniel Kahneman

Superficially, similar knowledge to what is in these 400+ pages can be found in a few Wikipedia entries. But you would miss out on the how and why cognitive biases and heuristics are so important. Medicine and research are bias-driven endeavors, and not understanding them is not knowing real-world medicine.

Only three? Yes. If anything, the two and a half months between mid-March and July 1st won’t be enough to read them all with the attention they deserve. But you should try.