Did ChatGPT-4 attend my lecture?

      No Comments on Did ChatGPT-4 attend my lecture?
Henry Trimen a.png

Henry Trimen, director of the Royal Botanic Gardens in Ceylon 1880-1896

Jim Clifford

In the lead-up to my take-home exam last April, I was trying to think of questions ChatGPT could not answer. I hoped that by focusing on details from my lectures that are not available on Wikipedia and other similar online sources, the large language model would fail to provide a strong answer. I was dead wrong:

Question: Why was Henry Trimen critical of J. E. Howard and his involvement in Britain’s efforts to introduce cinchona to Ceylon?

ChatGPT: Henry Trimen, a British botanist, was critical of J.E. Howard and his involvement in Britain’s efforts to introduce cinchona to Ceylon (now Sri Lanka) because he believed Howard’s approach was not based on sound scientific principles. [This is correct.]

Cinchona is a genus of trees and shrubs native to South America that produces quinine, an essential ingredient in the treatment of malaria. In the 19th century, the British wanted to establish a reliable source of quinine in their colonies, especially in malaria-prone regions like Ceylon. [This is correct, but not too surprising given you could find a Wikipedia article on the general topic of Britain’s attempt to introduce cinchona in South Asia.]

J.E. Howard was an English businessman and amateur botanist who took an interest in cinchona and its cultivation. He supplied seeds and plants to various British colonies, including Ceylon, with the aim of establishing cinchona plantations. However, Howard’s enthusiasm and ambition were not always matched by his botanical expertise. [This is spot on, and to my knowledge not a topic discussed in any online secondary source or peer-reviewed histories.]

Henry Trimen, who later became the director of the Royal Botanic Gardens in Peradeniya, Ceylon, criticized Howard’s approach for a few key reasons [The criticism came in Trimen’s first report after becoming director.] :

  1. Lack of proper identification and classification: Howard did not always correctly identify and classify the cinchona species he was dealing with, which led to the introduction of less productive varieties. [This was more focused on Trimen’s predecessor in Ceylon, but was a part of his concern with the whole cinchona project in the colony, which relied on Howard’s expertise.]
  2. Insufficient focus on local adaptation: Trimen believed that Howard didn’t pay enough attention to the importance of selecting cinchona varieties that were best suited to the local environment in Ceylon. This oversight led to the planting of species that were not well adapted to the region’s climate and soil conditions. [This is a little off the mark as Trimen was mostly critical of their obsession with finding the right species instead just testing the bark of young trees.]
  3. Inadequate attention to scientific research: Trimen felt that Howard’s approach was driven more by entrepreneurial ambition than by a genuine commitment to scientific research and understanding. This lack of rigorous investigation and experimentation may have limited the success of cinchona plantations in Ceylon. [I don’t know if he made this particular argument, but he had no respect for Howard’s scientific ability.]

Trimen’s criticisms reflect a broader debate during the 19th century about the role of science and scientific expertise in the colonial enterprise. While Howard’s efforts did help establish cinchona cultivation in various British colonies, his methods were eventually refined and improved upon by botanists like Trimen, who applied more rigorous scientific principles to the task. [Trimen actually shut down the cinchona project and helped the planters transition to tea. The Dutch succeeded by employing a chemist who tested the bark of year-old trees in Java and only planted trees with high cinchona concentrations on their plantations.]

This answer is not perfect, but it is surprisingly good for a topic that has little or no published historical analysis to my knowledge. I think it was close enough to get an excellent grade. I should note I’ve asked it the same and similar questions multiple times, and it does not always get it as close as the example above. In some cases, it hallucinates and makes up details about Howard working for the government directly, while other times, it focuses on Trimen’s concern with preserving Ceylon’s natural environment.

Where did ChatGPT find this answer? We are well beyond simple autocomplete generalizations here. Jim Webb wrote a chapter on cinchona in his environmental history of Ceylon plantations and he discusses Trimen’s efforts to salvage the cinchona project when he arrived in Ceylon and how he then “drew the cinchona experiment at Hakala itself to a close” in three short paragraphs, but does not mention Howard. Webb’s book is only available in snippet view on Google Books, so I doubt ChatGPT processed it. I’ve written about this in a draft of a chapter in a book that is years away from being published, but until now, it wasn’t posted online:

Dr Henry Trimen arrived in Ceylon as the new director of the botanical gardens in the early 1880s, at a time when the planters were struggling with the collapse of the coffee economy after the spread of rust and were starting to realize the cinchona they planted to replace it was bound to fail as well. He noted in his first report on cinchona that it was “to be regretted that circumstances should have thrown this very difficult genus into to hands” of people like Howard and Markham “who have had so little of the requisite training and experience in systematic, botany for dealing with it effectively”. Trimen went on to conclude “that however eminent a writer may be as a quinologist, a traveller, or a gardener, if he can see important botanical characters in the height of a tree, the chemical constitution of its bark, or the colour if its leaves, he is ipso facto disqualified to pronounce on questions of classification” (Trimen’s report is in the Kew Garden Archives).

GPT4 is a large language model created using a neural network, and it does not “know” the correct answer. Instead, the phrases “Henry Tremen”, “J.E. Howard”, “cinchona” and “Ceylon” were sufficient for it to find material in its training data that allowed it to produce a reasonably accurate answer some of the time. It has processed a lot of late nineteenth-century journals and other texts related to Ceylon, cinchona, economic botany and the botanical gardens from the Internet Archive’s Biodiversity Heritage Library and similar repositories, allowing it to formulate this answer from the primary sources. Perhaps this makes it more capable with nineteenth-century topics that are less hampered by paywalls (Ed Dunsworth’s examples make it very clear ChatGPT has not processed much recent scholarship and often defaults to bland, generic answers when probed about more recent topics).

Where does it leave historians? Clearly, we need to follow Mark Humphries’s advice on how to adapt our teaching to avoid assignments and exams it can master. I shifted gears last April and asked the students, “According to what we learned in this class, what three decades were the most important in the history of British industrialization? Justify your answer with references to readings and lectures”. At this point, ChatGPT has not taken my class or done the readings and can only provide a dull and predictable guess to answer this question. But its success makes me think it has the potential to become a powerful research assistant for historians, particularly if we can feed a large language model our curated collections of PDFs and archival notes. Thankfully, that is one of the many problems the open-source AI community is working on: https://www.llamaindex.ai/. Currently, these tools are focused on helping enterprises harness LLMs to process their data and I don’t think anyone is going to prioritize helping humanities scholars unless we engage with the technology and learn to use it ourselves. Thankfully, ChatGPT 4.0 is an excellent programming research assistant and can help us write Python code to link the Llama Index tools to our data. More on that in a future post.

Fake painting of failed cinchona plantations in Ceylon created with DALL-E.

Jim Clifford is an Assistant Professor in the Department of History at the University of Saskatchewan. He researches the urban environmental history of industrialization in Britain and its links with global environmental change in the nineteenth-century British world. He is a member of the ActiveHistory.ca editorial collective.

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License. Blog posts published before October  28, 2018 are licensed with a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License.

Please note: ActiveHistory.ca encourages comment and constructive discussion of our articles. We reserve the right to delete comments submitted under aliases, or that contain spam, harassment, or attacks on an individual.