❄️ DC public schools are back in session after two snow days and on one hand this is a relief — no one wants to make up extended snow days in the summer — but then most streets are still not plowed and have 0.5 lanes of traffic open making the morning drive a hazard. What is all this equipment for?

Two days ago I may have done some venting about peer review. Today I want to provide a solution: uber-peer review, by LLM.

The process is simple: as soon as the editor receives a manuscript and after the usual process determines it should be sent out for review, they upload it to ChatGPT (model GPT-4o, alas, since o1 doesn’t take uploads) and write the following prompt(s):

This is a manuscript submitted to the journal ABC. Our Scope is XYZ and our impact factor is x. We publish y% of submissions. Please write a review of the manuscript as (choose one of the three options below):

1. A neutral reviewer who is an expert in the topics covered by the article and will provide a fair and balanced review.
2. A reviewer from a competing group who will focus and over-emphasize every fault of the work and minimize the positive aspects of the paper.
3. A reviewer who is enthusiastic about the paper and will over-emphasize the work’s impact while neglecting to mention its shortcomings.

(the following applies to all three) The review should start with an overview of the paper, its potential impact to the field, and the overall quality (low, average or high-quality) of the idea, methodology, and the writing itself. It should follow with an itemized list of Major and Minor comments that the author(s) can respond to. All the comments should be grounded in the submitted work.

What comes out with prompt number 1 will be better than 80% of peer review performed by humans, and the cases number 2 and 3 are informative on it’s own. If the fawning review isn’t all that fawning, well that’s helpful information regardless. A biased result can still be useful if you know the bias! Will any of it be better than the best possible human review? Absolutely not, but how many experts give their 100% for a fair review — if such a thing is even possible — and after how much poking and prodding from an editor, even for high impact factor journals?

And how many peer reviewers are already uploading their manuscripts to ChatGPT anyway, then submitting them under their own name with more or less editing? What model are they using? What prompt? Wouldn’t editors want to be in control there?

Let’s formalize this now, because you can be sure as hell that it is already happening.

Much has been written and said about the faults of peer review but one thing I think hasn’t been emphasized enough so I’ll state it here: journal editors need to grow a spine. And they need to grow it in two ways, first by not sending obviously flawed studies out for peer review no matter where they come from, then by saying no to reviewers' unreasonable demands, not taking their comments at face value, and sometimes just not waiting 6+ months for a review to come back before making a decision.

📚 Finished reading: The Notebook by Roland Allen. It starts off strong, with an anecdote about the creation of the Moleskine brand, then goes in much depth about writing during renaissance and the enlightenment, topping it off with a few modern developments like BuJo. The chapters are self-contained and packed with information without being bogged down into too much detail — the Moleskine chapter is a good example of what to expect — at the expense of an overarching “story”. So, this is a collection of vignettes more than a systemic review and categorization of the types of notebooks through history, and that’s fine.

A few higlights:

• Michael of Rhodes, a 15th century member of the Venetian navy whose manuscript was the LinkedIn of the day: started of with personal interests, ended up full of useless (at best) and dangerously misleading information meant to impress future employers.
• Leonardo Da Vinci had no introspective passages in any of the thousands of pages he wrote down in his many notebooks. Janan Ganesh was right.
• The term commonplace arose from the commonplace book and used to mean a striking, exemplary passage in a book, play or speech one should write down and keep. Once commonplace books became popular and writers started including flowery passages with the express purpose of having them written down in the book the public got onto them and the adjective “commonplace” became derogatory. So it goes…
• The album amicorum or — and this may remind you of a certain online service — the friendship book made a similar turn, from an exclusive ritual of the educated few in 16th century Dutch universities to an 18th century fad among young women that was looked down upon by the patriarchy. What used to take two and a half centuries now happens in a few short years.
• The New York Times has always been evil; just read the chapter on Bob Graham’s notebooks to see why.

A more modest list for what I hope will be a more modest year:

1. Perhaps the Stars by Ada Palmer
2. Feline Philosophy by John Gray
3. Human Action by Ludwig von Mises
4. How Romantics and Victorians Organized Information by Jillian M. Hess
5. Project Hail Mary by Andy Weir
6. The Lincoln Highway by Amor Towles
7. Defeat at Gallipoli by Nigel Steel & Peter Hart
8. Leviathan by Thomas Hobbes
9. A Fortunate Man: The Story of a Country Doctor by John Berger
10. The Space Trilogy by C.S. Lewis
11. A Confederacy of Dunces by John Kennedy Toole
12. For Blood and Money by Nathan Vardi
13. The Billion Dollar Molecule by Barry Werth
14. Broken Stars by Various
15. The Armies of the Night by Norman Mailer

Remember, it’s the books you don’t read that count. And here are last year’s wish lists: 2024 — 2023 — 2022.

For all the hate X gets, you can still find nuggets of good information, Nassim Taleb and the Taleb-adjacent being a prime example. Here is one such post, from Juani Villarejo, shown here in its entirety for those who would rather not go to X to see the original:

Parkinson’s law says that work expands to fill the available time.

Jevons’s paradox states that every increased efficiency, will raise demand rather than decrease it.

And there is a work asymmetry:
Probably there are many more things you dislike doing than things you like.

Conclusion: If you allocate time to work, all the time will be filled with tasks to do.

If you make your work more efficient, your time will be filled with more tasks (demands increases).

But by the asymmetry, tasks you dislike doing have more chance to appear than tasks you like.

So when you make your work more efficient your time will always tend to be filled with more tasks you dislike doing.

Corollary: Never ever efficientize (sic!) the things you like doing. Take all the time and enjoy them slowly. They also serve as a defense wall against the things you dislike.

The links and emphasis are mine. For all its pretenses to the contrary X is still a horrible platform for anything longer than 300 or so characters and does not allow for hyperlinks.

A thought for the year, from the aforementioned Prof. Taleb:

Likewise, I don’t read letters and emails longer than a postcard. Writing must have some solemnity. Reading and writing, in the past, were the province of the sacred.

From How I Write, to which I have linked before. Good essays much like good books are worth re-rereading.

Here are a few links to start off 2025 (see if you can spot a pattern):

• Things we learned about LLMs in 2024 (ᔥDaring Fireball)
• The new Turing test for AI video… is absolutely horrifying (ᔥMR)
• The Ghosts in the Machine
• On skilled immigration

Happy New Year, dear reader!