Pyongyang on the Potomac
Yesterday, I quoted John Ioannidis’s description of how the pandemic was changing the norms of science:
There was absolutely no conspiracy or preplanning behind this hypercharged evolution. Simply, in times of crisis, the powerful thrive and the weak become more disadvantaged. Amid pandemic confusion, the powerful and the conflicted became more powerful and more conflicted, while millions of disadvantaged people have died and billions suffered.
First, note that no one person or thing (or a few countable ones) was responsible for the transformation, but rather that it was a natural property of the system. Then note that the pandemic sped up (or “hypercharged”) the system’s transition to its (breaking) endpoint. The American clinical trial ecosystem is even closer to its breaking point than science overall, and for the same reasons: its postulates make it untenable, and the influx of ZIRP-conjured pandemic-stamped money hypercharged the transition to this.
People go looking for ways to speed up clinical trials like they would for speeding up house construction. They ask: what is one key piece of legislation — the parking and zoning requirements of clinical research — that could either be tweaked or removed to break open the damn, so that trials would be as fast as they were in the 1960s? And why won’t the clinical trial hobbits tell us what those obstacles are, so that we can crush them?
But there is no one, or two, or two dozen such obstacles, the serial removal of which could speed up trial design, approvals, execution and readouts. The clinical trial path does not have boulder problem. Boulders would in fact be great: at least you can, while climbing over or making a 3-day detour around, fantasize about crushing them.
No, the problem with the path is that it runs through quicksand while being covered by a dense fog. Move too fast or stand in place and you die; move too slowly and you don’t get anywhere. Not that you know where to go: with all-encompassing fog you can only see five inches ahead anyway. There is not much time to think about anything else when you are in that kind of a quagmire, and how exactly would you imagine dealing with the fog? Drive it away with a big fan?
By “quicksand” I mean the moral, ethical and physical safety risks inherent to any clinical trail This is why every introductory course on the topic must start with the ignominious history of experimenting on humans… and the systems we developed to deal …immediately followed by a description of Institutional Review Boards and other regulatory matters. with them.
By “fog” I mean that much of the deliberation is not a matter of legislation but of opinion, values, principles and — the most loaded of words — comfort of people sitting on these IRBs and in the regulatory agencies. Depending on comfort level of individual members a question on whether a trial can proceed may take much deliberation or none at all. We can hardly know our own minds, and good luck about reading other people’s.
This is why I am skeptical that uncovering the FDA’s records would be of help to anyone but historians: the people who wrote them and interpreted them are no longer in play; different brains run the show, and the archives won’t reveal their thought process. Sure, sometimes there is one person sitting there that makes things worse than they should be, by being too slow, or thoughtless, or obstinate. But is “find better people” a tenable solution, when those people could be doing anything else, and with more reward of all kinds?
The pandemic hypercharge made everything worse. In addition to wading through quicksand while blinded by fog you now have to deal with many, many invisible neighbors elbowing you in the chest and kicking you in the groin, some intentionally, some out of carelessness, confusion, or not knowing any better. IRBs are overloaded and so are the regulators. What difference would any legislative change make?
Except, of course, it is to abolish one or both. It would lift the fog, sure, but would need a new kind of system to avoid making everyone sink to the bottom of the ethico-moral pit. Tearing down institutions is easy, building them is hard, as we see one but not the other being played out under our inattentive eyes.
All via Instapaper, which has been ever reliable in its role as a saucer.
Patrick Collison made a travel website:
It’s surprisingly hard to find good travel writing online. Upon landing someplace, you can peruse Wikipedia, but what else should one read? Below is a compendium of recommended pieces.
There is a single entry on “the Balkans” and it was fun to read if too simplistic. On the other hand, there are many references to Chris Arnade’s writing, so I hope that at least some of the other links are in that ballpark. (ᔥTyler Cowen)
Note: four of the five websites above are on Substack. I don’t like Substack. But it is so much of a behemoth that people you would least expect, like Nassim Taleb, are dipping their toes. The implications of even him abdicating to the winner of the most recent round of tech roulette are dire — yet another thing I should write about more, when time allows.
I nod my head agreeing with much of what Tyler Cowen says and writes, but the points where he is off are not minor. Here he is a few weeks ago, on a new RCT banning smartphones in the classroom showing (very) modest improvements in grades:
Note with grades there is “an average increase of 0.086 standard deviations.” I have no problem with these policies, but it mystifies me why anyone would put them in their top five hundred priorities, or is that five thousand?
He also points to an older trial from Norway, which had similar results. Cowen frames the bans as tiny gains for unknown and potentially enormous cost. And student comments like the following he found worthy enough to repost:
As an academically successful student in a pretty well ranked high school my recollection was that the entire experience was horrible and torturous and essentially felt like being locked up in prison. The pace of teaching was also so slow that the marginal value add of being in class was essentially 0 when compared to the textbook reading I would do after school anyway.
So… yes it was nice to have a phone and I don’t care if it distracts stupid students from learning.
And here is Rana Foroohar in this morning’s FT, under the headline Trump’s war on America’s schools:
[Randi] Weingarten, those of you reading outside the US could be forgiven for not knowing, is the head of America’s second-largest teachers’ union. In her new book, Why Fascists Fear Teachers, she lays out some of the history of authoritarian backlash against public education and its teachers, from the post-civil war Reconstruction era in the US, to Europe in the 1930s, to Vladimir Putin’s justification of crackdowns on teachers and universities in Russia (“wars are won by . . . schoolteachers”).
She also quotes the Canadian psychologist Bob Altemeyer, who found that a lack of “critical thinking” made people more receptive to authoritarian leaders. As he put it, “the very last thing an authoritarian leader wants is for his followers to start using their heads”. Or, as Trump so memorably put it after a 2016 primary win: “We won with poorly educated. I love the poorly educated.”
Reading is going out of fashion, but I would urge the student above, and Tyler Cowen, and everyone else who thinks eaking out marginal gains for top-performing students is worth the cost of “distracting stupid students from learning”, to (re)read Ursula K. Le Guin’s short story The Ones Who Walk Away from Omelas or — if they have more time — Dostoyevsky’s Brothers Karamazov which served as an inspiration with this passage in particular:
“I challenge you: let’s assume that you were called upon to build the edifice of human destiny so that men would finally be happy and would find peace and tranquility. If you knew that, in order to attain this, you would have to torture just one single creature, let’s say the little girl who beat her chest so desperately in the outhouse, and that on her unavenged tears you could build that edifice, would you agree to do it?”
And some may agree (I don’t)! But of course the equilibrium is not in focusing all of the world’s misery into a single person, as it tends to spread out, and you can’t lock up those exposed in a dank basement like the citizens of Omelas did. Rather, those people get to vote, and not in a way you may like.
Pyongyang on the Potomac
Two academics discuss science communication over BBQ and reach the wholly unoriginal conclusion that for increased trust in science, the American research community needs to:
These seemed redundant, as we have been marching towards more openness in science of every kind since at least the early 2000s. Would any scients be able to say, with a straight face, that their average peer projects more certainty, advocates for more gatekeeping, promotes reduced transparency and does not acknowledge controversy as much now compared to the 1950s?
One could in fact, if they were less charitable, blame this newly found openness for the collapse in trust. On one hand you see scientists fighting for clout on social networks, calling each other names, and blowing up small arguments — on the level of angels dancing on the head of a pin — into debates of the century. On the other, everyone and anyone, homeschooled child geniuses and crackpots alike, now has open access to much of specialized scientific literature, and to preprint servers for some samizdat science.
So maybe it is time to own it: yes, openning the kimono has lead to decreased trust in the estabilshment. But was that not the widely understood part of the bargain? I imagine Paul Feyerabend would have been proud of these recent developments.
How did the fellows above come up with the idea that more of the same would help shore up trust? Being academics, they have a reference — to the work of Sheila Jasanoff whose work on “civic epistemiology” is described thusly:
Jasanoff’s research identifies distinctive features of how Americans evaluate scientific claims:
Public Challenge: Americans tend to trust knowledge that has withstood open debate and questioning. This reflects legal traditions where competing arguments help reveal the truth.
Community Voice: There’s a strong expectation that affected groups should participate in discussions about scientific evidence that impacts them, particularly in policy contexts.
Open Access: Citizens expect transparency in how conclusions are reached, including access to underlying data and reasoning processes.
Multiple Perspectives: Rather than relying on single authoritative sources, Americans prefer hearing from various independent institutions and experts.
But of course this is hopelessly outdated, if it were ever true to begin with. Jasanoff herself cautions in the chapter of her book “Designs on Nature” where she describs the concept, that the framework offers conceptual clarity at enormous risk of reductionism, as it does not account for differences across social strata, through time, etc. The book is from 2005 and the research it is based on is even older. The “Americans” described above no longer exist.
Jasanoff’s civic epistemilogies were tied to countries. In the last twenty years these countries have lost ground as unifying social forces to a variety of cultures and subcultures. Her descripton of 2005 America may today better apply to the upper-middle-class across a subset of countries more so than a single nation. In each country, the different epistemiologies are becoming more and more opposed. How could we possibly trust each other?
There may be no way to return the trust in scence to the 20th century levels. But if we were to try, the most obvious method would be a return to gatekeeping. Leave the science to the scientists and let the outcomes speak for themselves. Keep all discord inside conference halls and university cafeterias. Show more decorum and respect, if grudging, to every scientist colleague while being more discriminatory of who is “a scientist”: PhDs from recognized universities only, please.
This would, of course, be a step back and I in no way, shape or form condone a turn of events quite like this — least of all because it would exclude me from the conversation.
Is there a way to stick to the “open science” principles while keeping some modicum of community trust? Being a fan of Costco, their sort of low but effective barrier to entry is appealing. For the uninitiated: Costco charges a modest annual membership ($65, or $130 for their “executive tier”) for the privilege of shopping for premium and premium-mediocre products at incredibly discounted prices. Their only profit is from the membership, as there is little to no margin. But then they also don’t need to spend money on things like advertising, keeping the shelves pretty, or monitoring for shoplifters.
The space between payinh $65 per year and earning a PhD is vast. Whatever the new gate is, it should probably not be degree-based. Maybe have it be a professional society that also takes up interested laypeople using its own criteria. Or a verified subscription to Experimental History. Whatever it is, make it official, make it publich, and make it stick. Then, keep most of the conversation inside the circle. Keep all ambiguity inside the tower, please, just make the tower entrance bigger and charge for entry.
Is this the way? I am not sure. Maybe science doesn’t deserve the public’s trust and attempts to increase it are like plugging tiny holds on a massive damn about to burst. But to those who care, let this be some food for thought.
Of course, every profession has its linguistic pearls.