I’ve recently had to do a simple but tedious literature search: finding the incidence and prevalence of a handful of rare diseases, using only peer-reviewed articles as reference. The perfect job for AI, some would yes. Well, yes for an AI, but apparently not for LLMS.

I am a proponent of using LLMs as tools of the spellcheck sort, but they are still not ready to act as serious research assistants. Bing presented me with gobbledygook that ended with a repeating string of “KJKJKJKJ” or some such — and no, my prompt was not for it to pretend a cat was walking over the keyboard — while Bard gave a beautifully formatted table full of inaccurate numbers and fake references. Correcting it would have taken more time than doing the research myself.

Which is to say: I anticipate a lot of angst when incoming students start passing off unadulterated LLM work as their own in anything that’s not a business writing course.