With 23andMe closing shop today and the bluebird bio sale to private equity last month it is clear that the DNA bubble has burst.

Every bubble leaves something positive in its wake. Yes, there was a lot of speculation with tulips in the Netherlands, but the Netherlands is still the world’s top exporter of cut flowers. There was a railway bubble in the United States that left us with a lot of railroad tracks and not so great passenger rail. More recently, the dot-com bubble left decent network infrastructure and a lot of IT professionals with nothing better to do than to invent Web 2.0.

And so with DNA. Sequencing has never been cheaper, and it does have some valid uses. Unfortunately, there are many harms of fetishizing DNA, from thinking that DNA mutations are the be-all and end-all of every disease pathology — think, “the fat gene” — to completely missing the point of the entire field of epigenetics, which has much more to it than molecular changes to histones and base pairs.

Business and finance are now the first to realize that there is more to genetics than DNA, and more to medicine than genetics. Academia and funders, ossified as they are, will be slower on the uptake and come to this epiphany one retirement at a time.

Earlier this year I mentioned the anonymous X account Crémieux as a proponent of the concept of “National IQ”. Now we know the person behind that account, and the truth is in fact quite boring. This is the line between having a pseudonym and creating a sock puppet. (↬Sasha Gusev)

Today’s FT essay on the rise of the anti-vaccine movement was a miss. Instead of asking why so many people lost trust in institutions it goes straight to politics: 10 paragraphs on Germany’s AfD, no mention of whether some of the people’s concerns were valid. With that, the movement can only grow.

Also on that other site: Grok 3. I wish I could say it was pure hype and bragging from a man-child in need of attention. Alas it’s the real deal: it gave me less BS, and faster, to my now standard set of querries. Brave new world, etc.

The “Real World Risk Institute” — RWRI — is Nassim Taleb’s answer to the question of what his Incerto would look like if it were a course. The twentieth workshop starts on July 7 and lasts for 2 weeks. This is what I wrote on that other site in response to the announcement:

Strongly endorse. Took the first one in July 2020 and if it weren’t for it I’d still be a federal employee on a visa. It’s not the knowledge you get (you have the Inecrto (sic!) for that), it’s the thinking

And I meant every misspelled word. Go if you have time: scholarships are available, math is not required.

The headline: “Cheap blood test detects pancreatic cancer before it spreads”.

The reality:

The nanosensor correctly identified healthy individuals 98% of the time, and identified people with pancreatic cancer with 73% accuracy. It always distinguished between individuals with cancer and those with other pancreatic diseases.

The 98% number means that two out of 100 healthy people who take the test would have a false positive result. It also misses more than 20 out of 100 people with cancer, giving them a false sense of security. If used in a mostly healthy population — a reasonable assumption to make for a screening test — a positive result would more likely than not be a false positive, and yet you would still miss plenty of actual cancers.

These are abysmal assay characteristics and the test should never see clinic, but you would never know it from the headline. (↬Tyler Cowen)

One query in, I am convinced of the value of Deep Research and think it is well worth the $200 per month. The sources are real, the narrative less fluffy, the points it makes cogent. The narrative review is not dead yet, but it is on its way out. Here I am thinking about those reviews that are made to pad junior researchers CVs while they are introducing themselves to a field, neutral in tone and seemingly comprehensive in scope. There will always be a place for an opinionated perspective from a leader in the field.

In a year, the AI algorithms went from an overeager undergrad to a competent graduate student in every field of science, natural or social. Would o3 make this into a post-doc able and willing to go down every and any rabbit hole? Even now two hundred dollars per month is a bargain — if the price stays the same with next generation models it will be a steal.

The one snag is that it is all centralized, and yes the not so open OpenAI sees all your questions and knows what you want. For now. Local processing is a few years behind, so what is preventing nVidia or Apple or whomever from putting all its efforts into catching up? How much would you pay for your own server that would give its in-depth reports more slowly — say 30 minutes instead of 5 — but be completely private? And without needing benefits, travel and lodging to conferences or any of the messy HR stuff.

The brave new world is galloping ahead.

(↬Tyler Cowen)

It has been more than six years ago now that Nassim Taleb rightfully called IQ a pseudoscientific swindle. Yet this zombie idea keeps coming back, most recently as a meandering essay by one Crémieux who, through a series of scatter plots and other data visualizations, attempts to persuade that “National IQs” are a valid concept and that yes, they are much lower in South Asia and Subsaharan Africa than the rest of the world.

This hogwash prompted another series of exchanges on IQ ending, for now, with this X post that recapped some points from Taleb’s original essay for a lay audience. That alone is worth reposting, but what I thought was even more interesting was one of the replies:

But I still prefer my doctor or pilot or professor to have an iq of over 120 (at least). I am sure it matters. Not as the only characteristic, but still.

While missing the point so completely that it wasn’t worth replying to, the post is a good example of another IQ-neutral human trait, to hypothesize on properties in isolation without considering nth-order effects. Let’s say your surgeon’s IQ is 160. What are the implications for their specialty of choice, fees, where they work, and bedside manner? Are they more or less of a risk-taker because of this? Does their intellectual savvy transfer more to their own bottom line, picking high-reimbursement procedures over a more conservative approach? Even if you said “all else being equal I’d prefer someone with a higher IQ”, well, why would you if everything else was equal? In that case would it not even make more sense to pick someone who did not have the benefit of acing multiple choice questions based on pure reasoning rather than knowledge? And yes, Taleb wrote about that as well.

Another set of replies was on the theme of “well I don’t think we could even have a test that measures IQ”, showing that they don’t know what IQ is — it is the thing measured by an IQ test. There is some serious confusion in terms here and X is the worst place to have a discussion about it, everyone shouting over each other.

Finally, since I agree with Taleb that IQ as used now is a bullshit concept, people may surmise as they did for him that I took the test and that I am now, disappointed in the result, trying to discredit it. I do think it’s BS for personal reasons, but of a different kind: some 25 years ago as a high school freshman in Serbia I took the test and was accepted to mensa. Having attended a single, tedious meeting in Belgrade shortly afterward I saw that the whole thing was indeed laughable and haven’t thought about it again until reading that 2019 essay.

Having a high IQ means you are good at taking tests, and correlates with success in life as much as your life is geared towards test-taking. There is nothing else “there” there and good test-takers unhappy with their lives should focus on other of life’s many questions, like how to execute a proper deadlift and whether home-made fresh pasta is better than the dried variant.

The founders of statistics were a bunch of drunk gamblers so I’m used to seeing parallels between games of all kinds and science. In that vein, Andrew Gelman writing about cheating at board games made me think of cheating in clinical trials, only the right parallel there would be cooperative games in the style of Pandemic and the many Arkham/Eldritch Horror games made by Fantasy Flight.

These are tough games — Eldritch Horror in particular — where players and rule keepers are one and the same and all on the same team. And they are easy to beat if the players are willing to fudge a rule here or re-roll the dice there. And to be clear some of the rules are punishing including — from my absolute favorite, Arkham Horror: The Card Game — The Grim Rule:

If players are unable to find the answer to a rules or timing conflict in this Rules Reference, resolve the conflict in the manner that the players perceive as the worst possible at that moment with regards to winning the scenario, and continue with the game.

The Bonferroni correction for multiplicity could be a form of TGR, as could sensitivity analyses with particularly harsh assumptions. Note that TGR is a shortcut intended to enhance enjoyment of the game. Sure, your investigators may be devoured by Cthulhu or all go insane, but even that is more fun than 30 minutes spent looking up and cross-checking arcane footnotes in thick rule books, or worse yet trawling through Reddit for rule tips. Note also that clinical trials and science are not there (only) for fun and enjoyment and that applying TGR has consequences more serious than having to restart a game.

Which is to say, it pays off to be more thoughtful about your analyses and assumptions when designing a study and think — am I making this assumption and doing things this way because I don’t want to fool myself, or because the alternative would take too much time? The same goes for assumptions that are too lenient, but of course you already know that.

Two days ago I may have done some venting about peer review. Today I want to provide a solution: uber-peer review, by LLM.

The process is simple: as soon as the editor receives a manuscript and after the usual process determines it should be sent out for review, they upload it to ChatGPT (model GPT-4o, alas, since o1 doesn’t take uploads) and write the following prompt(s):

This is a manuscript submitted to the journal ABC. Our Scope is XYZ and our impact factor is x. We publish y% of submissions. Please write a review of the manuscript as (choose one of the three options below):

1. A neutral reviewer who is an expert in the topics covered by the article and will provide a fair and balanced review.
2. A reviewer from a competing group who will focus and over-emphasize every fault of the work and minimize the positive aspects of the paper.
3. A reviewer who is enthusiastic about the paper and will over-emphasize the work’s impact while neglecting to mention its shortcomings.

(the following applies to all three) The review should start with an overview of the paper, its potential impact to the field, and the overall quality (low, average or high-quality) of the idea, methodology, and the writing itself. It should follow with an itemized list of Major and Minor comments that the author(s) can respond to. All the comments should be grounded in the submitted work.

What comes out with prompt number 1 will be better than 80% of peer review performed by humans, and the cases number 2 and 3 are informative on it’s own. If the fawning review isn’t all that fawning, well that’s helpful information regardless. A biased result can still be useful if you know the bias! Will any of it be better than the best possible human review? Absolutely not, but how many experts give their 100% for a fair review — if such a thing is even possible — and after how much poking and prodding from an editor, even for high impact factor journals?

And how many peer reviewers are already uploading their manuscripts to ChatGPT anyway, then submitting them under their own name with more or less editing? What model are they using? What prompt? Wouldn’t editors want to be in control there?

Let’s formalize this now, because you can be sure as hell that it is already happening.