The Archive is Not a Toy: The Hidden Problems of a ‘Vintage’ AI

Jacob Polay, Chloë Farr, and Jessica Jack

Generations of social historians have scoured archives and invented new ways of doing history through their collaboration with other fields. A new “vintage AI” model, talkie, is an example of what happens when knowledge extraction replaces collaboration.

Parsing the various communications around the model, the team behind talkie has framed their new LLM in two ways. In their blog post introducing this new model, and in an interview with the podcast Hard Fork, the team behind this project describe its predictive potential. They aim to test whether language models can predict events beyond the period of the training data, and do so by training it with pre-1931 documents. The team presupposes that if this “vintage” model can predict its future, then modern models trained on the current web can predict our future.

talkie chatbot interface, however, does not engage with the model as a predictive tool. There is no proof of its efficacy. In our testing, their local model is not shipped with any predictive capabilities, nor do they provide sufficient documentation to achieve any prediction. Instead, it acts more like a toy to interact with a particular version of history. Their tagline advertises the model as “an LM from 1930,” as if the user is talking with an entity from the past. The framing between prediction and history disguises a tool whose use is unclear and whose underpinnings are flawed.

These flaws begin with the data on which talkie was trained. Its training corpus was a subset of the published English-language record before 1930. Like most western archives, this is a corpus likely dominated by white, male, and elite writers. Other voices are outweighed in the records and further filtered by digitization. The result is a model that speaks confidently for the past in the voice of its narrowest slice.

The talkie team made these choices deliberately. On the podcast Hard Fork, talkie team member Dr. David Duvenaud, associate professor at the University of Toronto, stated they “wanted to make everything entirely publicly available in open source, and 1930s is the most recent date that has almost zero legal headaches.” However, the team has not released the training data sources, so what little we know about the data set is based on their public statements. The training dataset seemingly contains only print-origin English-language texts, excluding hand-written or multilingual sources. Their blog post specifically lists “books, newspapers, periodicals, scientific journals, patents, and case law” as their training corpus. They drew these materials from repositories like Harvard University’s Institutional Data Initiative, the Internet Archive, and products from the Common Pile project. In so doing, the team relied on data that was easy to find and ready to use, instead of developing a representative corpus with diverse perspectives from the past. A concentrated reliance on ready data can produce problems, like reproducing narrow-minded points of view held by certain people in the past.

This reliance on easy data was not from a lack of access to expertise. Five months before talkie‘s release, one of us, Farr, met with Levine and Radford for an hour-long discussion about optical character recognition (OCR) and accessing historical archives. As part of the talkie project, state-of-the-art OCR is in development, intersecting with Farr’s research on vision language models for archival transcription. However, instead of discussing new AI-enabled transcription tools as Farr had expected, the team questioned her about how to source training data for their forecasting model. Farr’s questions about their corpus curation methods, such as what would be included or excluded, and how the gaps would be acknowledged, were met with only vague interest. They considered these questions out of their present scope. Five months later, the model shipped with the corpus problems Farr had flagged represented only as ‘limitations’ and ‘future directions’ in the release post, which acknowledges her contribution in ‘helpful discussions’.

Heeding warnings like Farr’s may have lessened some of the problems with talkie’s outputs. As seen in Figure 1, talkie produces bigoted responses then frames them as a representation of 1930; an outcome of the project’s methodology. The hard work of expanding the archive is being subverted. Humanities researchers have read marginalized experiences against the published record, mapped enslaved people’s lives using colonial ledgers, and reconstructed Indigenous histories from oral traditions and the margins of colonial documents. The talkie team’s use of easy and accessible documents, and the model’s resulting bigotry, overshadows the professional and collaborative efforts that have gone into diversifying and expanding the archives beyond these problematic voices. These efforts have grappled with the archival challenges ignored by talkie: much was never printed; much of what was printed is not digitized; most digitized handwritten sources are not transcribed. The ephemeral and multilingual sources needed to train a model on anything approaching the full archive of the past simply do not exist in machine-readable form.

Framing talkie as a chatbot dictates how the public will use it. The chatbot has largely been received as a toy. In one Reddit thread, users repeatedly prompted it to predict Hitler’s rise. A chatbot ‘from 1930’ invites users to ask it what 1930 did not yet know. The interface produces this behavior and the model’s construction makes such use inherently fraught.

Without historical framing, a casual user of talkie has no reason to read “‘[A woman] is inferior to a man intellectually” as anything other than a broadly-accepted historical truth, instead of it being a perspective held largely by the white elite. This is a direct result of talkie’s team taking minimal steps towards preventing users from interacting with problematic results. Standard industry approaches like post-training guardrails are common for language models trained on bigoted internet-scraped data. The talkie team characterized these post-training guardrails as “put[ting] their thumb on the scale,” ignoring how they already tipped the scales through their dataset choices.

The talkie team is well equipped to perform post-training on their model. talkie was built by a group including Dr. Duvenaud, whose research focuses on AI governance and ethics in AI use, and who previously worked on Anthropic’s Alignment Science team; Alec Radford, a former employee of OpenAI and principal author on GPT-2, CLIP, and Whisper; and Nick Levine, a former quantitative researcher and financial historian at the hedge fund Winton. Levine has an MPhil on the history, philosophy, and sociology of science from Cambridge and has published on “information overload in postwar America”.

This team has expert knowledge and experience, but talkie represents a minimal application of this expertise in relation to the bigoted output of the chatbot. Their only apparent safeguard, added post-release, is an additional model (Qwen3Guard family) that reviews the chat inputs and outputs and flags content “as potentially inappropriate”, obscuring the output with a “Show it anyway” button (see Figure 2). An accurate warning would instead explain that the content is likely incomplete and is, at best, a narrow perspective of the past.

talkie’s poor representation of history is a side effect of a larger problem. The model was conceptually built as an experimental tool to see if language models can accurately predict the future, also known as “forecasting.” The blog post assertively claims the models predictive capabilities, yet is devoid of evidence enabling researchers to reproduce their experiment. The version that could serve serious research is gated behind compute, and is the same version released with no guardrails at all.

As we have mentioned, we have no proof of the predictive capability of talkie. However, the team’s framing of talkie as a predictive tool aligns with an explosive growth in the gambling and prediction markets in North America. In February of 2026, the night before the United States struck Iran, hundreds of bets correctly predicting the strike appeared on Polymarket. The New York Times reported on these betting markets, raising questions about asymmetric information shaping these markets. Prediction markets are now a multi-billion-dollar industry built on the geopolitical forecasting the talkie team wants to test. A sufficiently capable model in this lineage would be valuable to anyone whose decisions depend on prediction, including the markets, and would therefore be very attractive to investors. As we have seen with the rise of language models like ChatGPT and its competitors, the developers do not get to decide how the tools they develop are used once released into the world. talkie is similarly positioned.

Whatever their intentions, the team should release the training data and co-design these models with archivists and historians rather than using them as beta testers. The team says they intend to work with historians on future post-training and model development. Yet, they already had that chance and failed to use it.

In releasing this chatbot, the team has set aside Farr’s warnings, and what humanities researchers have spent a century working out. Documents are only partial records of the past. Archives preserve the records those in power choose to keep. Absence in the record is itself evidence. Reading those absences requires interpretive expertise the model does not have and the interface does not invite. Modern models are far more than statistical mimics, but they remain tethered to their inputs. They cannot represent perspectives or facts outside of the dataset. If we want complete answers, we must build complete foundations.

Jacob Polay is a PhD student in History at the University of Saskatchewan, studying the roles Large Language Models have in the historical method. His current research involves creating an information retrieval pipeline using artificial intelligence tools to unlock the early modern archive at scale.

Chloë Farr is a PhD student in Computer Science at the University of Victoria, working at the intersection of artificial intelligence, archives, and digital humanities. Her research focuses on large-scale text recognition and analysis of historical documents, including newspapers, maps, and archival records. Learn more about Farr’s work on GitHub.

Jessica Jack is a PhD student in History at the University of Saskatchewan, developing applications for Large Language Models in historical research. They are doing so through studying settler land use in late 19th century and early 20th century Saskatchewan.