October 24, 2023

The Get Shorty style of peer review

Serbia in the 1990s had a peculiar mass media landscape in that movies were rarely officially released, yet were shown on TV days after premiering through the magic of pirating, practiced by both broadcast and cable networks. The most successful of these was TV Pink, now a horror show of reality TV, and one of the many peculiar features of TV Pink was that its daytime content would rely heavily on those E! making-of fillers and patter interviews with exhausted celebrities on their movie-promoting circuit. While home-bound sick kids of America filled their days with Bob Barker and Jerry Springer, in Serbia it was all Hollywood all the time — unless you were the weirdo who liked to watch reruns of 1960s kids shows and their poorly made hyperinflation-era remakes, which was the only thing state TV was capable of producing.

But that wasn’t me! So when I got bacterial pneumonia back in 6th grade and was stuck at home for two whole weeks while receiving twice-daily intramuscular right-into-the-gluteus aminoglycoside antibiotics — the lackadaisical attitude of Serbian pediatricians towards dosing and toxicities is a different story — Pink took up more of my time than I care to admit, and making-of videos from that period got engrained in my memory more so than the movies themselves. Topping the list was the 1995 comedy Get Shorty starring John Travolta, which back then I thought must have been the biggest blockbuster ever if they were talking about it so much.

All this is a preamble to what I heard said by Travolta, or his co-star Danny DeVito, or maybe it was the director Barry Sonnenfeld, and it was this: the movie was based on a book, and the book was outstanding and written by Elmor Leonard who had a way with writing dialogue, and whenever they had an urge to improvise their lines they would hold back because Leonard must have already thought about the things that came to the actors minds first and decided that, no, this thing on the page was better.

And I have heard that line so many times — my pre-frontal cortex still developing — that I have now completely internalized it and act on it unconsciously. When evaluating someone’s work — outside of grading papers, for that is a different matter entirely — I start with the assumption that they have thought long and hard about the paper they’ve submitted for my peer review, certainly longer than the few hours I can dedicate to reviewing it, and I give them the benefit of the doubt. My first impression is probably something they thought about and dismissed, to come up with what they are submitting. Now, some papers are so egregiously wrong that they will still be red all over after I’m done; but if there is a small difference of opinion, or a nit to pick with style, or something I would maybe have done slightly differently, I just let it be, in deference to the authors’ work and respect to their, the editor’s, and — why hide it? — my own, time.

After being on the receiving end of quite a few paper and grant reviews myself [Note: Oh, and meetings. So many meetings. ] , I am beginning to suspect that not everyone is following the Get Shorty ethos.

Now, the worst peer review I have ever received was also the shortest. It was for a paper about a clincial study in rare disese that had a one-sentence rejection from the first journal where it was submitted: “Only 11 patients, they need more”. But most other reviews are not “bad” in that sense, but rather overly verbose and nit-picky about the tiniest of details with dozens of comments per review, the purpose of which is not to improve the article, but rather to show to the editor of the prestigious journal — the higher the impact factor, the more nits to pick — that the reviewer was worthy of the invitation to provide his or her services free of charge to the academic publishing machine. Look at me, ma’, I’m paying attention!

Which is fine for papers, I guess, since the reviewers will be in the ballpark of your field (those that aren’t won’t accept the review), and you may at least get a chance to respond. Grants are worse: not only are the reviewers forced into it for the prestige of being on a study section, there is little chance if any that they will have the knowledge of your field [Note: This is why, I suspect, the best predictor of receiving an NIH grant is already having received an NIH grant. Not only have you been stamped as a success for the “educated lay-people” on the study section, but if you reapply to the study section their knowledge of your field will have been what you told them, and the Program Officer, in prior grant applications. ] The problem is trebled if you apply with a clinical trial, because all the people with clinical trial expertise across all of the NIH study sections could probably all fit in a Mini. But how much more difficult could designing a clinical trial be from running a lab, eh?

The examples are many and I’m not at liberty to discuss most of them, but back when I was opening trials in T-cell lymphoma a grant was not funded mostly because they didn’t think we could enroll patients for this “rarest of the rare” disease in time (we were well over half-way done with enrollment by the time we heard of this decision). In case you were wondering why all the money goes to breast, colon, and lung cancer research, well, no one ever had a problem recruiting for those!

There is a role for peer review: to weed out the impossible and the truly un-fundable. But after that it may as well be a lottery: why would rolling the dice be worse than adding up laundry lists of small, irrelevant issues that could sink an application which gets assigned to two or three particularly detail-oriented reviewers. It would also save a hell of a lot of time for everyone involved.

October 23, 2023

So if not daily world news, what then? Well, Axios Local is a good option for DC, and has daily newsletter for more than two dozen other cities. StreetSense is a DC-only enterprise, and more relevant for me than whatever this is from The Washington Post. If the Council does start handing out vouchers to support local news, I know where mine will be going.

📺🎶 The soundtrack of Only Murders in the Building continues to be our commuting go-to. All of the faux-musical numbers are now available, and listening to our 11-year-old sing along to the tongue-twisting Pickwick Triplets… is beyond delightful.

When a project I have been putting off for 2 months gets done after 90 minutes of concerted effort, how should I feel: relieved or annoyed? Because right now it’s the latter.

After a nearly two-year hiatus, Wondermark is back. But will there be a calendar?

October 22, 2023

Convulsionews

Here is an obvious analogy for you: the physical world — meatspace, if you will — as “meat” of an actual body, both skeletal (muscles, ligaments, tendons and such), and visceral (entrails, the liver, vital organs); the internet as nerve impulses connecting the various parts both sensorially (how are the navels of the world doing these days?) and in effect (from Facebook groups to GoFundMe pages bringing actual change).

You know how X and other social networks made everything feel connected to everything else? Well, there is an organic counterpart to this phenomenon, and it’s called a generalized tonic-clonic — or grand mal — seizure, manifesting, in the clonic phase, in widespread convulsions of the body.

The reason why our bodies are usually not convulsing is that the nerve impulse pathways are tightly controlled in space: there are separate nerves, differentiated brain areas for different roles, and let’s not forget the biggest separation of them all: two semi-independent brain hemispheres connected only by the corpus callosum which, imagine this, is sometimes cut completely for treatment of refractory seizures. There is also chemical separation: many of the pathways are inhibitory, and the most abundant neurotransmitter in the body is not dopamine, serotonin, acetylcholine or others you’ve heard of because they go haywire, but glycin, a modest amino acid which people don’t hear about because it is so good at its job of tamping down bad impulses.

The world’s ongoing convulsions started — after an initial tonic phase — right after we have all become interconnected: Hezbollah, Hamas, and your neighborhood association all hooked up to the same firehose. There is a feeling at the edge of my consciousness that the answer to solving them is in ourselves, and not in a new age self-fulfilment way but in pragmatic steps we can take to extrapolate from this most obvious analogy.

October 21, 2023

Just another day in Maryland.

Photo of a man in antique clothing carrying a bold eagle, standing in a muddy arena surrounded by spectators.

October 20, 2023

Education as scaffolding

Adam Mastroianni had another interview with Russ Roberts, and this one didn’t sit with me as well as his previous appearance. They talked about things we learn in school — higher education, for the most part — and what the point of it all was when most of what we cram in is forgotten.

They gave several examples of this, all of which I have quickly forgotten (ha!), but listing things we were made to learn in medical school never to use again was a popular past time on #medtwitter so I will list a few topics that come to mind first:

Though that last one was clearly a relic of Serbia’s socialist past, the first four weren’t, and are still being taught in pre-med courses and medical schools around the world. If the goal was to have every doctor know all of these throughout their careers, well, mission failed. But why would we even want that to happen?

Well, I have come around a bit since that 6-year-old tweet and came to appreciate the exposure to different concepts as the scaffolding to whatever career we end up in. No one cries out, after a skyscraper is complete, about all the money and time wasted putting up a scaffolding, setting up cranes, temporary elevators, and such. It is not a perfect analogy since most people in higher education don’t have a blueprint — not even medical students since a doctors’ job can be anything from an artist (plastic surgery) to woodworking (orthopedics) to glorified administrative asssistant (general practitioners in most countries) — so it is like building a scaffolding to nowhere, parts of which ossify into the building proper, parts of which decay with time, and parts of which you dismantle as soon as it seems safe to do so, since you hate them from the bottom of your being.

“Why ever did I bother learning about the Krebs cycle five different times!?” [Note: Twice in high school — biology and chemistry separately, and three times in medical school — chemistry proper, biochemistry, and physiology. ] I cry out now, as a hematologist/oncologist without a regular practice; but things could have taken a turn towards a career in organic chemistry, or genetics, or one of those specialties where the cycle is more relevant (though really oncology may very well be one of them!)

The poor Krebs cycle is notorious because it is repeated so often without practical use for most of medicine, but there are many more such concepts throughout life that went in one ear and out the other (Are viruses causing hemorrhagic fevers made of DNA or RNA? Well, I knew it for my USMLE Step 1 exam!) [Note: RNA, says Wikipedia. ]

The scaffolding analogy puts a slightly different spin on grades as well, which could be a rather useful signal of where your construction should go and what kind of a building you want to make, and not your worth as a human being that most teachers and some students want it to be. But let’s not bring up grades again.

So I was surprised by Mastroianni’s and Roberts’ surprise about us forgetting — because of course we do! And if the intent of the teachers was to instill knowledge that will last forever and ever, well, most of it is a miserable failure, except for that one sliver of insight that each of us carry for life. But the slivers are different for each of us, and to appreciate your unique sliver you may still need background knowledge that you will eventually forget — the more specialized the area, the more background knowledge needed, so good luck trying to untangle that web.

Tim Harford:

The proportion of genetics papers with autocorrect errors was estimated in 2020 to have reached 30 per cent. The Human Gene Name Consortium decided to rename the genes in question, wisely accepting that this would be easier than weaning researchers away from Excel.

At the intersection of science and technology lies the festering boil that is Microsoft Office.

News noise

Nassim Taleb in Antifragile: [Note: The link points to an excerpt posted on the Farnam Street blog, which I stopped following years ago — too much noise in the form of wisdom nuggets — but still has its uses. You should really read the whole of Incerto ]

The more frequently you look at data, the more noise you are disproportionally likely to get (rather than the valuable part called the signal); hence the higher the noise to signal ratio. And there is a confusion, that is not psychological at all, but inherent in the data itself.

Now let’s add the psychological to this: we are not made to understand the point, so we overreact emotionally to noise. The best solution is to only look at very large changes in data or conditions, never small ones.

Alan Jacobs today:

If you’re reading the news several times a day, you’re not being informed, you’re being stimulated. Try giving yourself a break from it. Look at this stuff at wider intervals, and in between sessions, give yourself time to think and assess.

Always good to see convergence on important topics. I now get most of my news from books.