This answer is not perfect, but it is surprisingly good for a topic that has little or no published historical analysis to my knowledge. I think it was close enough to get an excellent grade. I should note I’ve asked it the same and similar questions multiple times, and it does not always get it as close as the example above. In some cases, it hallucinates and makes up details about Howard working for the government directly, while other times, it focuses on Trimen’s concern with preserving Ceylon’s natural environment.
Where did ChatGPT find this answer? We are well beyond simple autocomplete generalizations here. Jim Webb wrote a chapter on cinchona in his environmental history of Ceylon plantations and he discusses Trimen’s efforts to salvage the cinchona project when he arrived in Ceylon and how he then “drew the cinchona experiment at Hakala itself to a close” in three short paragraphs, but does not mention Howard. Webb’s book is only available in snippet view on Google Books, so I doubt ChatGPT processed it. I’ve written about this in a draft of a chapter in a book that is years away from being published, but until now, it wasn’t posted online:
Dr Henry Trimen arrived in Ceylon as the new director of the botanical gardens in the early 1880s, at a time when the planters were struggling with the collapse of the coffee economy after the spread of rust and were starting to realize the cinchona they planted to replace it was bound to fail as well. He noted in his first report on cinchona that it was “to be regretted that circumstances should have thrown this very difficult genus into to hands” of people like Howard and Markham “who have had so little of the requisite training and experience in systematic, botany for dealing with it effectively”. Trimen went on to conclude “that however eminent a writer may be as a quinologist, a traveller, or a gardener, if he can see important botanical characters in the height of a tree, the chemical constitution of its bark, or the colour if its leaves, he is ipso facto disqualified to pronounce on questions of classification” (Trimen’s report is in the Kew Garden Archives).
GPT4 is a large language model created using a neural network, and it does not “know” the correct answer. Instead, the phrases “Henry Tremen”, “J.E. Howard”, “cinchona” and “Ceylon” were sufficient for it to find material in its training data that allowed it to produce a reasonably accurate answer some of the time. It has processed a lot of late nineteenth-century journals and other texts related to Ceylon, cinchona, economic botany and the botanical gardens from the Internet Archive’s Biodiversity Heritage Library and similar repositories, allowing it to formulate this answer from the primary sources. Perhaps this makes it more capable with nineteenth-century topics that are less hampered by paywalls (Ed Dunsworth’s examples make it very clear ChatGPT has not processed much recent scholarship and often defaults to bland, generic answers when probed about more recent topics).
Where does it leave historians? Clearly, we need to follow Mark Humphries’s advice on how to adapt our teaching to avoid assignments and exams it can master. I shifted gears last April and asked the students, “According to what we learned in this class, what three decades were the most important in the history of British industrialization? Justify your answer with references to readings and lectures”. At this point, ChatGPT has not taken my class or done the readings and can only provide a dull and predictable guess to answer this question. But its success makes me think it has the potential to become a powerful research assistant for historians, particularly if we can feed a large language model our curated collections of PDFs and archival notes. Thankfully, that is one of the many problems the open-source AI community is working on: https://www.llamaindex.ai/. Currently, these tools are focused on helping enterprises harness LLMs to process their data and I don’t think anyone is going to prioritize helping humanities scholars unless we engage with the technology and learn to use it ourselves. Thankfully, ChatGPT 4.0 is an excellent programming research assistant and can help us write Python code to link the Llama Index tools to our data. More on that in a future post.