Posts in: science

Sunday links, clinical trials edition

Drugs which look great in those cellular machinery flow charts with boxes and arrows pointing every which way, and which may even cure a few genetically monstrous and wholly artificial lab mice, tend to flop where it matters. Lowe links to 11 such examples and writes in more detail about the twelfth.

An overly long article ᔥNintil with which I don’t completely agree For example, Kroetsch describes the role of a site investigator as resembling “that of a glorified data entry clerk - the investigator’s primary responsibility is gathering the data that the drug company needs and sending it to them”. This is incorrect: site investigators usually have clinical research coordinators and data managers to do it for them. But this deserves a post of its own. but which nevertheless provides a good overview of the many things wrong with how clinical trials are being conducted in the US, the biggest one being that they are reinventing the wheel each and every time they are done. The “lean trial” proposal at the end matches my own thinking.

Teslo picks up on the tech bro magical thinking streak in which things you don’t sufficiently understand seem eminantly solveable using the most recent technological developments. Five years ago it was electronic medical records and blockchain, now it’s clinical trials and AI. The article gives the many reasons why things are not that simple. Now, if we all agreed on the set of LLM prompts that would provide an unbiased protocol and informed consent form review thus eliminating as many people from the loop as possible, well, then we may be on to something.

If someone qualifies for euthanasia, should they also not be eligible for every expanded acces, compassionate use, right-to-try scheme imaginable? Obviously: yes. Maybe not so obviously: there is a branch of my subspecialty aptly named desperation oncology which in the vast majority of cases leads to false hope, financial ruin and, worst of all, time misspent in doctors' offices and infusion clinics instead with your loved ones. As a doctor and a human being I am partial to life, so I see state-assisted dying programs like Canada’s MAID as monstrous, but “you’d rather be dead so here, take this drug” is only a half-step above that qualifier and leads to the bad reputation of experimental therapies.

Monday links, science, technology and cults

Friday links, quick hits

  • Martha Lane Fox: The Price of initiative just collapsed. On the lag between invention and implementation, from printing press to AI.
  • Scott Shambaugh: An AI Agent Published a Hit Piece on Me. A taste of what’s possible, with follow-up.
  • Steve Blank: You Only Think They Work For You. About contracted service providers. The article is about public relations, but applies just as well to contract research organizations (CROs) and these complex dynamics are part of the reason clinical trials are so expensive.
  • Andrew Gelman: The 80% power lie. A clear example of why frequentist statistics, so favored by conservative regulators, institutional review boards and scientific review committees, are more often than not based on false assumptions.
  • Dr. Drang: Chinese New Year and Ramadan. The pseudonymous doctor often shifts between programming and scripting languages, from R through Python to Mathematica which coincidentally matches my own amateur programmer journey. For this post he used Emacs Lisp and I am very much interested in Emacs and Lisp right now due to recent experimentation with Linux. Saliency FTW.

Where is the Delaware of clinical trials?

I admire the fight for clinical trial abundance, I truly do, but the longer I think about it the more convinced I am that making nips and tucks to federal law and FDA guidance is the wrong way to go about it.

I have been involved in clinical trials for more than a decade and have seen it through the eyes of an investigator, sponsor and IRB member directly, and owing to a few friendships with ex-FDA employees, from the eyes of regulators as well. The common thread between all cases of delay and all frustrated attempts to speed things up were not laws and regulations but humans. The blankfaces and boxcheckers. Those IRB members who prefer the list of possible toxicities in a consent form in table form as opposed to a list, or a list as opposed to a table, and with percentages of expected occurrences instead of qualitative statements of possibility, but can you please explain what does percentages mean in qualitative terms, and is it “toxicity” or “adverse event”, and you should list only true toxicities not hypotheticals, except when we require you to add each potential adverse event — sorry, toxicity — whether it happened in this or other trials or not, and for each round of these changes there is another 3-week turn of the IRB roulette during which a different person may view the edits and give their own two cents — leave their own fingerprint.

And that is only the IRB! Their are personal imprints to be made at every level of clinical trial desing and implementation, between different departments of the Sponsor, “key opinion leaders" Or “KOLs”. The whole KOL ecosystem is worth writing about at length, just not today. sitting on advisory boards, administrators and worriers-in-chief at clinical trial sites, and even, rarely, clinical trial investigators themselves although in the US at least they are the ones least involved in clinical trials. More often than note, the way to leave your fingerprint is by citing a rule or a law or precedent from the FDA or the IRB or some other state-level regulatory agency whether or not said rule, law and/or precendent apply.

To be clear: most people in the clinical trial ecosystem are not like this. But much like airport noise complaints, in which one household accounted for almost 80% of all complaints, it only takes a handful of people to gum up the works and make it appar like the whole system has conspired against smooth trial execution.

So ho do we achieve abundance? Cutting down on the laws that govern trials is an option, and would certainly reduce the degrees of freedom at which blankfaces and boxcheckers operate. But America is a country of min-maxers and I can easily see things going sour with drive-by trials and phase 1 clinics being run out of a storage unit in South Florida. Which operate even now, so just you wait for the walls to come down and the minmaxing trial beast to be unleashed. Chesterton’s fence and all that.

Could we not reduce the number of gummer-uppers in the chain instead? There are plenty of trials that are quick to start, quick to execute, and not even all that costly, largely due to people being reasonable. Can we inject more reasonable people into the ecosystem, if we can’t inject more reason into people who are already there?

This thought brought me to Delaware, the second smallest state of the USA that has 60% of Fortune 500 companies incorporated there owing to the convenience, flexibility and predictability of its corporate law system. To my layperson’s eyes it seems, from the example of Delaware, that its not the law itself that makes the difference but rather who implements it and how — which was on spectacular display earlier this week.

Convenience, flexibility, predictability.

Now clearly the Delaware of clinical trials can’t be Delaware — it is far too small and wouldn’t have enough trial candidates. But California, Texas, Florida and New York could all vie for the spot, each having a population similar to Australia, the phase 1/2 promised land. Texas was the first to make some waves, tiny and of an uncertain direction, but in the ballpark of what would be needed. The only changes that federal law needs are those that would allow the states to compete among themselves in being clinical trial champions.

If that last image made you think of China and its competing provinces, it is for a good reason: the Chinese have been eating everybody’s lunch for the past few years, in large part because of the vicious internal competition elaborated on elsewhere. There are other lessons there, for some other time.

Tuesday links, on science, medicine, technology and a bit of something extra

Why no scientist should hang their hat on a single pet theory, with real-world examples. The same problem haunts the world of biotech even as its denizens claim their superiority at drug development.

About a recent Nature Medicine article which found that LLMs were no better than Google at helping patients diagnose and manage their self-reported maladies. The reasons are those that I suggested two and a half years ago — ChatGPT can give you the correct answer from a properly structured clinical vignette, but the art and science of medicine are transferring the reality in front of you — the patient’s haphazard story, their hodge poge of medical records, the subtle physical exam findings — into something salient. Not saying AI won’t get there at some point, but it clearly still needs work.

Rao has collected 101 (!?) of his best Twitter threads and a few hundred single tweets into a book. A note on the title page says:

This book is LLM-friendly. Point your LLM to venkateshrao.com/twitter-book if you want it to explore it. A full interactive archive, explorable via an AI oracle, is under development.

Living up to his call to be (slightly) monstrous.

Yes, it is a person I hate making a good point, which is that the brutalist architecture of L’Enfant Plaza is out of place so close to the National Mall and should be kept where it belongs. I even prefer the proposed neoclassicist style to what Trump’s ego would want, which I imagine to be a Dubai And even Dubai would be better than what’s in the President’s id. on the Potomac.

A rare day-job update: we have not one but two papers out in the journal Nature Medicine this morning. The first is clinical and the other biomarker data from the same randomized trial, both open access. The last big paper was more than two years ago, and the post-publication feeling hasn’t changed.

Monday dive into social media, with a brief note on life after Twitter

  • Jason Kelly on X: The crisis in biotech startups is not just “biotech being cyclical” - you can see clearly that the rise in Chinese startups is not cyclical over the last 25 years - it’s spiking up in the last 10 years (see chart below from @AsimovPress). This is in response to a post from Bruce Booth arguing that the rise in Chinese biotech is not just a threat for the US but also an opportunity, and one that should be a cause for optimism. Booth responded in turn. I am close to finishing Apple in China and based on that alone I tend to side with Kelly. Riding dragons is a dangerous business, as both Apple and Tesla have found out in their respective industries. There are, of course, a few ways in which biotech is significantly different from cars and phones that requires some more thinking, but that is for a different post.
  • Cory Doctorow on Mastodon: On December 28th, I delivered a speech entitled “A post-American, enshittification-resistant internet” for 39C3, the 39th Chaos Communications Congress in Hamburg, Germany. This is the transcript of that speech. Not just post-American but post-Chinese Internet as well, so this is a talk about robustness and dare I say antifragility through decentralization, not anti-imperialist rambling (although this being Doctorow there is some of that sprinkled in too). A video version is also available.
  • Matthew Dowd on X: Pickle Expose: Sliced and Diced. Engrossing article in the form of a full-blown X post, with links, in-line images and half-decent typography. (ᔥJohn Gruber))
  • Katyayani Shukla on X: Warren Buffett literally gave a free 1-hour masterclass on business. I saw the video and thought that Mr. Buffet was looking unusually spry for a 95-year-old! Well, the speech is from July 18, 2001 and is available in both YouTube video and transcript form, so there was no need for contextless X posts with worse quality video and audio. This particular one got 7.5 thousand likes and 2 thousand reposts in less than two days. I guess not everyone appreciates context and citation as much as I do. (ᔥJohn Mandrola, also on X)

Note: Despite three of the four links being from X, I have to admit that I am finding Mastodon more and more enjoyable and the superior of the four post-Twitter offerings, at least for me and my tastes. I am still vacillating on whether I should just use my micro.blog account to follow all non-X users, but then Ivory is too good of an app not to use. Advice appreciated.

Friday links, science and biotech edition, with extended commentary

The case for faster bench-to-bedside-and-back type of research, with which I agree. It is remarkable, however, how each generation interested in biomedical research reinvents the wheel without checking prior art. I would also argue strongly that the (correct) thesis of the essay is not a refutation of the biotech-as-casino hypothesis but rather its confirmation, unless you enlarge “biotech” to include academia and government research but then what are we even doing. Investors have no patience for nuance and view clinical trials as dichotomous regardless of how companies try to present them, and interpreting translational research results requires even more patience and tolerance of ambiguity.

Ginexi has been a program at the NIH for more than two decades, so caveat lector, but many POs are indeed mini-Moseses in their scientific domains. On one hand they perform important and valuable work, on the other the importance of a single human being to the careers of investigators young and old tend to favor those with soft skills of communication more than those of scientific and intellectual rigor. No judgements on my end because I genuinely can’t tell if the alternative would be any better.

Some genuinely good advice on how to write grants in a way to increase the odds of them being funded, with emphasis on accepting the reviewers' comments and suggestions and approaching the grant resubmission as one would an offer to revise and resubmit a scientific manuscript, with much thanking and back-bending. Do keep that in mind when you read the next item.

This is true for most, as there are far too many academic right now for all of them to have soul in the game. However, academia continues to ask for more than it gives back out of too many people, while at the same time putting a negative selection pressure against people who are stubborn, single-minded and thus predisposed to a soul-in-the-game phenotype (see above). The only reason why the system survives at all is that the churn has been too low to fully reveal the tension, but it continues to creep towards the breaking point providing yet another case study of things that happen gradually and then suddenly.

Mid-week links, with extended commentary on some

A story of white male millennials being blocked from career advancement because of DEI. The fields he highlights are scripted television shows, news magazines and academia which aren’t exactly thriving now but per Savage did back when these policies were being implemented (early to mid-2010s). The rise of the “manosphere” and crypto brotherhood was therefore revenge of the jilted, which sounds plausible. One does not become an NFT peddler because they want to but because they couldn’t fulfill their life-long dream of being a tenured Women’s Studies professor.

Note that only early-career positions seem to have been affected, where people with no skill and/or time to choose among many qualified candidates decided to simultaneously switch from one discriminatory heuristic to another. So maybe not everyone should have done it at the same time (a good policy to follow for any change)? Would a method for unbiased selection of early job candidates have to involve an AI? And what are the demographic of OpenAI and Meta’s leadership again?

A Y Combinator company tries to use machine learning to discover new drugs. No, they didn’t figure it out and are now pivoting to selling pickaxes instead of digging for gold themselves. Godspeed.

Retelling of the story of penicillin’s discovery and mass manufacturing, which is much more complex than the typical serendipity-is-important (or, sometimes, luck-favors-the-prepared-mind) tale that begins and ends with Alexander Fleming’s accidentally contaminating a bacterial culture with mold. This is not to disparage the more popular variant: a big part of my childhood was soaking up wild tales of invention via Discoveries Unlimited which originally came out in the year of my birth but was dubbed to Serbian and played on repeat on state TV in the early 1990s. Of course, my own children now have something infinitely more majestic than the “Video Encyclopedia” from that show… and use it to play Roblox.

This also took me back! And not only because of Dune, which I played several times through the end and liked much more than the sequel, one of the first real-time strategy games. No, this article is also about It came from the desert and Sid Meier’s Pirates! and many other games that used short-but-sweet bursts of different mechanics to tell a coherent story, which is qualitatively different from a collection of mini-games sold to highlight the multi-functionality of Nintendo’s new controller. I hope an indie game studio somewhere is working on bringing them back.

Thursday links, for the academics