As a fan of Ruxandra Teslo’s writing — 25 mentions to date! — it pains me to write that her recent article in “Works in Progress”, for which she shares the byline with Open Evidence chief product officer Amol Punjabi, had me wince about a half-dozen times too many to ignore. Worse yet, I agree with the thrust of the article: that China is eating America’s lunch in cell and gene therapy and will soon come for the rest of biomedicine. Heck, that is one of the main reasons I am soon going back to clinical medicine, seeing too many business flights to Shanghai and Beijing time zone Zoom meetings in my future [Note: In case you were wondering, the correct number of each for me personally is exactly zero. ] had I continued down the industry path.
Alas, Teslo, Punjabi and whichever LLM did their research had cut too many corners on the way to the largely appropriate destination. Let’s count a few of them.
The old, cheap generic chemotherapy drugs still rock. A combination of two or three chemotherapy drugs developed in the 1970s and 80s is still the gold standard for treating testicular cancer. Chemotherapy has tamed what was once a pancreatic cancer-level death sentence into a diagnosis that doesn’t even have a “stage IV”. Speaking of pancreatic cancer: daraxonrasib, the K-Ras inhibitor which Teslo just a few weeks ago deemed a turning point, [Note: That article was, however, still of much higher quality than the one discussed here. Or maybe I don’t know as much about K-Ras and pancreatic cancer as I do about CAR T-cells and myeloma. Could this be a case of reverse Gell-Mann amnesia on my end? ] doesn’t even come close to what bleomycin, etoposide and cisplatin did for testicular. I guess they don’t make turning points like they used to.
The transformation of oncology started long before mid-2010s. The article paints a simplistic picture of oncology’s history. First there was surgery, followed by, in the 1890s and the discovery of X-rays, radiation therapy. Blunt and unsophisticated chemotherapy which relies purely on the cancer cells’ propensity to divide faster than non-cancer cells came in the 1940s and 1950s. Finally, in the mid-2010s, after we learned more about the molecular biology of cancer [Note: I guess that, if you wanted to show off your academic status, you should use a 10-dollar word like “underpinnings” here instead of the plain, grade school level “biology”, much in the same way you should find and replace every “use” with “utilize”. But that would, of course, make you a 10-dollar ass. ] came “immunotherapies” by which the article largely means CAR T-cell therapies in general and one in particular, ciltacabtagene autoleucel, known to friends as cilta-cel (generic name) or Carvykti (brand name and the one used throughout the essay; this is telling).
Look, I am no fan of Siddharta Mukherjee’s but at least his history of cancer, The Emperor of All Maladies got the sequence right. Rituximab, a monoclonal antibody which some still consider the original immunotherapy — after all, it acts mainly by siccing patients’ own immune cells and complement towards the target lymphoma and leukemia cells — was approved in 1997 after a Phase 1 trial that started in 1994. Trastuzumab, another monoclonal, was approved for Her2-positive breast cancer in 1998. Imatinib, a revolutionary wonder-drug which inspired dozens of me-too small molecule competitors, had its first-in-human study in 1998 and was approved just three years later, in 2001. Each needed just 3 years to get from the very first patient being dosed to FDA approval; remember that factoid it may become relevant in a few paragraphs. These were actual cures for lethal, aggressive cancers. But if the narrative is that China has accelerated the development of the first true advancement in cancer cures since the advent of chemotherapy let’s just pretend they don’t exist.
Myeloma treatment is not as brutal as painted. Although, of course, everything is in the eyes of the beholder, or rather the mind of the patient having to suffer through it. I do take issue with all three of the specific side effects that the essay highlights, as well as the time burden of myeloma is described. To wit:
Patients come in and out of the clinic for injections, take pills at home and undergo repeated blood tests, living according to a calendar organized around treatment days and recovery days. They also have to contend with the side effects of the medications. Dexamethasone can produce a sleepless agitation followed by a physical and emotional crash. Bortezomib often damages peripheral nerves, causing tingling and a burning pain in the hands. Daratumumab often leads to immune suppression, leaving patients more vulnerable to infections.
Dexamethasone is given in bursts and, thanks to the decidedly non-industry funded trials led by S. Vincent Rajkumar, at a much lower dose than before, minimizing these sorts of side effects. Similarly, bortezomib is now given less frequently and in different ways (under the skin instead of intravenously) to minimize nerve damage. And if you think immunosuppression is bad with daratumumab, well, try wiping out every antibody-producing cell in your body then waiting until you can get all of them back, and yes that includes needing to receive all your childhood vaccines again.
Separately, repeated blood tests are a sine qua non of multiple myeloma management, or really of any cancer management, even after a “cure”. If we aren’t monitoring for recurrence of the primary disease we are fussing over other cancers which may or may not be the result of the treatment itself, or of a person’s general propensity to have cancer. [Note: In fact, two biggest risk factors for having cancer, other than a genetic mutation/hereditary syndrome, are age and prior personal history of cancer. ] And yes, that goes even for patients whose CAR-T treatment leads to durable complete remissions. Especially with CAR-T treatments which are known to cause cancer.
Speaking of which, cilta-cel/Carvykti is not a walk in the park either. Cytokine release syndrome (CRS) and Immune effector cell-associated neurotoxicity syndrome (ICANS) are two particularly nice side effects of all conventional CAR-T therapies, Carvykti included. They are frequent and severe enough that most patients need to be treated in the hospital and be within driving distance for the next four weeks. Many end up being admitted to the critical care unit. CAR-Ts that target BCMA, like Carvykti, also cause profound immunosuppression (vide supra) and require patients to repeat their childhood vaccination series. Carvykti, however, is in a league of its own as on top of all that it can also cause Parkinsonism. This is not to throw shade at CAR-Ts, they truly are revolutionary. But let’s not condemn other myeloma treatments for their toxicity when the alternative is worse in some ways, about the same in others.
BCMA CAR-Ts are, for most patients with multiple myeloma, not a cure. The essay cites 12-month results of the CARTITUDE-1 trial, where 76% of participants who received the cells [Note: But not including those enrolled to the trial who never got them, whether because they couldn’t be made, they were too sick to get them, or just plain died. This is how you play the denominator game. ] had no signs of myeloma at 12 months. Quote:
But what happened afterwards is perhaps even more striking: in the Abecma progression free survival curve, the line falls continuously. By contrast, in Carvykti, the line starts to plateau. Extended follow-up at five years confirmed that 33 percent of Carvykti patients remained disease-free.
This is false: there is no plateau. Figure 2 of the NEJM article describing these results has some numbers at the bottom not included in the Works in Progress essay. These represent the “number at risk” — participants who were still available for follow-up at a given time point; others have either progressed, resulting in an unwanted “drop” in the curve, or have not yet been followed for that long [Note: There are actually more reason for a participant to be marked as a “tick” without dropping the line, i.e. to be “censored”, some more nefarious than others. For a good primer on this “informative censoring” see, for example, this article ] and are marked with a triangle here though more commonly they are merely a tick. The “plateau” is an artifact of too few participants getting to 24 months, only 9. It completely disappears in extended follow-up, with the curve continuing its descent at and past 24 months in Figure 2A, all the way to 60 months where a cluster of vertical tick marks precedes yet another mirage of a plateau, again with only a handful of patients being at risk. Let’s pray it ain’t so but I suspect that, if we were to continue following these participants to 10 years, the curve will continue going down and down and down.
You could make the same story about artefact plateaus about daratumumab as well. It, too, has been pushed up all the way to first-line treatment and even before overt disease; concerns about longer follow-up needed for what is usually a slow-burning disease remain. Compare and contrast to imatinib in CML in this recent essay from Vinay Prasad, who concludes with:
There is progress in both diseases but more in CML. CML is more clearly a success story. There is much room for progress in myeloma. Myeloma is not yet curative, sadly. Presenting survival over time is misleading and masks more complicated narratives.
Carvykti’s approval timeline was not gobsmackingly fast. Most misleading is the side-to-side comparison of the Chinese cilta-cel and its American predecessor, ide-cel development pathway. Ide-cel includes the development of CAR T-cells in general (1989–2012) and the first BCMA targeting proof-of-concept (2013). Cilta-cel emerged from Zeus’s head in 2014, like it didn’t require both CAR-Ts to be developed and BCMA to be validated as a target. The tag line of the figure is that “China’s BCMA CAR-T reached FDA approval just 11 months after the US, despite starting decades later.” Hogwash.
Note the development timelines: ide-cel’s first-in-human study started in 2014. [Note: I should know: I was there! Funnily enough I was the in-house fellow on call on the days when two of the first 3 participants received their cells and had the honor of escorting them to the intensive care unit that very night. Both had both their myeloma and all of bone marrow wiped out in the process. ] It received FDA approval in 2021, for a total of 7 years of clinical trials. Cilta-cel’s first-in-human was in 2016 with a 2022 approval; 6 years. Let’s finish up our mini-mental test: how long did it take for the FDA to approve rituximab, trastuzumab and imatinib, from the first patient dosed?
These are only the highlights, but going much deeper would be nitpicking. I don’t know whether this amount of laxity with the truth was intentional, but the essay is almost as misleading as a Seattle lady’s GPS, taking her straight onto light rail tracks. Once there, you can only go in two directions: forward, towards loosening up regulations to match China’s Wild West, or backwards, tightening up regulator requirements for Chinese assets and trials and punishing companies for doing business there. [Note: See how I ties going “forward” with less regulation and “backward” with more. This can easily be flipped to portray less regulation as going backwards, but I leave doing that in full as a fun exercise for you, dear Reader. ] What happened to going sideways? Diagonally? Up or down? What if it the time to approve revolutionary cancer treatments has doubled because the follow-ups aren’t as revolutionary? And then get drowned out further by the me-toos and the ghost drugs which make much better competitors in the biotech beauty pageant, where whom you know and where you came from is more important than the increasingly pliant, malleable and quicksand-appearing ground truth?
But sure. China.
Back in Pleistocene when I was in grade school and still had dozens of hours per week to spend on PC games and when computing power and storage space were so precious that even quick-save — forget about autosave as the feature was yet to be even a glimmer in game developers’ eyes — were just not a thing, my obsessive tick which I repeated every 10 or so minutes was to stop whichever game I was playing at the time, go to the menu, and save my spot, “just in case”. This is when saving the game carried weight and you could name each time point, all of mine being named just that: “just in case”, or rather, its abbreviated Serbian equivalent “ZSS”. I rarely ever reloaded since these were mostly Sierra and LucasArts-style point-and-click adventures in which it was quite literally impossible to get stuck or make a wrong move, or early RPGs like Lands of Lore which, OK, had its challenging moments, but certainly not enough to warrant 10-min saves.
Fast forward three decades, when my own children have in that regard fallen so far away from the tree that they’ve landed on the gaming Moon
[Note:
Fallen upward, clearly. There is nothing wrong with this metaphor. Please carry on.
]
where the abundance of autosave has destroyed any spot-saving reflex they could have possibly had and with it also any sense of dread for things to come. I was reminded of this stark difference after JTR wrote about his own fears playing Subnautica. There is a link there but it leads to a “Page Not Found” even though the article came through in the RSS feeds and I don’t know if this was a feed-only post or a Micro.blog bug so
[Note:
Mystery solved, please see @jtr’s comments below.
]
I will quote liberally here:
The first time I played, the game had the element of surprise. I remember my first reaper: it came out of nowhere and grabbed my Seamoth like a plaything. I yelped, slammed the Alt+F4 keys, and stomped out of my room as white as the hallway wall I was leaning against, mumbling “oh my god” over and over. Now I know better. I know where they are, I can see them in the distance, and… I’m still scared. But I go ahead anyway. The fear is not pushing me away; it’s teaching me to be prepared. The only thing that’s really scary is fear itself.
See, I never played Subnautica but if I did my playing style would be not much different from JTR’s. Not so for my (nearly) seven-year-old who zips over and across and around reapers and dies and loses some of his supplies and shrugs his just-out-of-toddlerhood shoulders and gets at it again. And if you think that’s because he’s too young to know better you haven’t seen his older sisters play a time challenge level of Astro Bot, a game which severely punishes any hesitation, unnecessary pausing and haphazard jumping.
This is one way of many ways in which the kids are better than their parent, and I put much credit in the autosave abundance!
The power of the Internet is that, under an innocuous title such as Sdcv-quick Update, on a mostly technical blog dedicated to Emacs, one can find a most delightful essay by James Somers — from way back in 2014, the days of still-capitalized Internet — about the power of the old Webster’s Dictionary, how it outshines its modern successors, and how, wonder of wonders, you can download and install Webster’s Revised Unabridged Dictionary (1913 + 1828) onto your computing device of choice. [Note: But not Emacs. That’s where quick-sdcv.el comes in! ]
Somers does not quite reach the heights of David Foster Wallace’s Authority and American Usage but then he also takes only about a fifth of the space to make his point. [Note: If these two weren’t enough, “Draft No.4” by John McPhee will do nicely to meet your dictionary essay needs. ] His whole blog makes for great reading, most of it having been written pre-LLMs. This is important: Somers is a professional writer whose most recent articles in The New Yorker and The Atlantic keep glazing AI. If you think I am exaggerating, here are some of the more recent titles: “The Coming Software Apocalypse”, “The Scientific Paper is Obsolete”, “How Will A.I. Learn Next?”, “A Revolution in How Robots Learn”, “The Case That A.I. Is Thinking”… With this kind of coverage, who needs a marketing department?
📚Still reading Inventing the Renaissance, and Lorenzo de Medici’s brother Giuliano of course had a mention. Still, I wasn’t prepared for his bust being quite so metal.
This is at the (not Smithsonian!) National Art Gallery in DC, which has a rich collection of renaissance works.
📚 Some 18th century childhood doodles in a 17th century William Shakespeare first folio, as seen in the Folger collection. Kids will be kids!
In the preamble to his Morose thoughts at the Semiquincentennial, @ReaderJohn notes:
I’m on a social medium (I refuse to abuse the plural “media”) with an astonishing number of people, many of them decades younger than me, who manage, without coming across as idiots (au contraire: I’m struck by how many there make me feel unobservant and thick-skulled about what I do observe), to focus on positive, and personal, and local things. Kudos to its designer, who consciously designed it that way (I’m not sure how, except that one never knows how many people follow him or her, and there are no buttons to simply “like” a post).
That last parenthetical is, I believe, exactly the reason why micro.blog turned out the way it did. My first thought was that it filtered out people who liked to see numbers go up — many of them not of the clearest mind — right at the outset. But that is not all there is to it, probably not even the most important part. The intentional lack of statistics cuts the feedback loops which tend to make some people into complete assholes, and every person into an occasional asshole. [Note: Or at the very least an asshole-appearing online presence, but to the exposed person — meaning you, dear reader — there is no difference. ]
Every popularity contest will reward the extremes. This is why I gave up following the Bear Blog Discovery feed. Random posts from to-me unknown authors just popping into my RSS reader [Note: These days a combination of the [Inkwell][4] Android app on my Daylight tablet and my own homebrewed [Inkling for Emacs][5], which is where I’m writing this! ] reminded me too much of Twitter’s algorithms, and even Bubbles — posts from 5,000+ independent blogs, including this one, ranked by timeliness and popularity — favors criticism of AI and tech in general combined with outrage/despondence/resignation towards news of the day/breakage of everyday life/civilizational decline. The only ever Infinite Regress post that ended up on the Bubbles front page fits right in. [Note: A kind reader even uploaded it to Hacker News, where it — thankfully! — received just 4 upvotes and no comments. Small mercies. ] It is, in that sense, no different from Reddit: the medium (of voting) is the message.
Incidentally, these Bubbles and Hacker News and Kagi Small Web and indieblog.page and ooh.directory visitors all leave footprints on this here web site’s Tinylytics dashboard, which has become delightfully uninterpretable owing to the influx in the past few months of what I can only assume are digital ghosts from Hong Kong, Singapore, China and Mexico, in that order. An unexpected benefit of LLM crawlers.
There is something about numbers that makes people’s brains stop working. This is common in medicine, where reflexively treating a lab abnormality without thinking an iota about the patient or even about the ground truth — is this number here “real” or is it a blood collection/lab analysis/data entry error? — is a phase most doctors go through and some never leave. Call it video game brain: confusing the hardcoded information of an RPG stat or a FPS health bar with more malleable values we get from physical measurements.
Well, I know enough about myself not to expect an effortless change in behavior. The effort tank being depleted daily by issues more pressing, I avoid having to interpret these numerical tricksters in any way I can. You see, for feedback to be of any use there has to be effort somewhere and by making leaving it effortless (thumbs up or down? how good was our service from 1 to 10? the text field is optional!) we have made interpreting it seemingly straightforward but in fact harder. Did someone “like” a blog post because reading it was a life-changing experience? Slightly more amusing than the cat photo just below? A toilet seat mistap? Or was it herding?
Now think about all those feedback surveys you started filling because the first page was a deceptive 1–10 scale only to abandon it because page 2 had five large fields for free text, all with a mandatory character count. This puts majority of the cognitive effort on the feedback provider; reading it does take more time than glancing at a number, but the receiver can quickly and effortlessly tell whether it is a) from someone whose opinion they care about and b) what the said opinion is.
So yes this is a long-winded way to nudge you towards writing more emails. Or leaving more comments. Or even starting your own blog. More words, fewer numbers, please. And yes, yes, I am aware how silly asking for more words sounds in these, our Days of Slop. But to go back to the blog post that started all this, and then two links deep to a most brilliant text from Sam Kriss: when everyone from your middle manager bosses to Guardian journalists to prize-winning authors and random tech folk debase themselves with AI, the value of the human-written word does in fact go up.
Have a great weekend!
Doc in a Box from Alex Tabarrok links to an official state government document, from the Utah Department of Commerce. The document is titled “Key Statistics on the Doctronic Pilot Program” but reads more like a bulleted press release, full of percentages without a denominator, begging for a flow chart. Press releases are like that because you typically won’t add images — although this one randomly selected from today does indeed include it along with the full abstract submitted to the ASCO annual meeting, and good for them — but more importantly because you want to pick the best possible picture-perfect view of your shiny spotless data elephant without also acknowledging that it has a rear end, a bunch of flies buzzing around, smells a bit rank. Does your elephant not have an ass, Utah? Or did you just copy/paste what Doctronic — a startup whose wonky web page doesn’t even work — sent you?
Doctronic.
We have hit a technical snag. Go to Homepage to hit it again.
So how many patients could they have evaluated? This article in JAMA Forum says that “[p]hysicians hired by Doctronic will review the AI’s output for the first 250 patients before the system takes any action and will review the next 1000 patients retrospectively after the AI agent begins acting autonomously.” Are the key statistics from the first 250? The very first bullet point in the press release summary document says that the program is still in Phase One and that “the number of patients so far is limited”, so I guess not. Is it 100 at least? Surely they wouldn’t use a percentage as high as 97 if there were fewer than that involved. Except that as low as 30 will give you a percent roundable to 97. So, 30 to 249?
Why am I being so pedantic? Well, these techniques are par for the course in biotech world but coming from a state agency make me think there is a bit too much enthusiasm for it, coming from a government source. Compare and contrast to the shellacking LLMs got in this report from the Office of the Auditor General of Ontario, which reviewed AI Scribe functionality from 20 vendors. Their report even has absolute numbers in it! These state government officials should realize that they are prime targets for flim flam merchants and should behave accordingly.
Note that I am not against the idea in general. The project’s goal is in fact quite noble: there is no reason why plain ol’ machine learning shouldn’t be able to suss out majority of refill requests for chronic medications and flag patients who haven’t had their bloodwork or diabetic foot assessments done, or who’ve had abnormal office blood pressure readings at prior visits. Having that easy refill option available would mean a patient coming in for an in-person visit for what should be “only” prescription refills is even more of a signal that something else may be amiss, even if the patient can’t or won’t verbalize it. So yes, LLM refills, bring ’em on. Doctronic’s end-goal of actual autonomous Shoggoths putting on white coats and replacing MDs, PAs, NPs and other credentialed humans… not so much.