Critical AI and Large Language Models for Humanists
Instructors:
Anouk Lang
Duration: 2nd week
Detailed description
The course will begin with a historical perspective on text generation technologies (eg. ELIZA, Markov chains), leading on to GPT-2 and GPT-3 experiments and what they can tell us about genre. We then move to considering training corpora and what goes into building them, and how the idiosyncrasies of training data shape machine learning outputs. Participants will then be introduced to word embeddings and the notion of multidimensional vector space as a way of capturing the diversity of contexts in which a word or token appears. We will look at examples of digital humanities research that uses word embeddings to probe the relationship between concepts and the discursive environments in which they circulate, and participants will be able to try vector algebra for themselves using pre-trained models of historical texts. We then turn to the question of ‘bias’ in LLMs, beginning with a critique of the notion of debiasing, and using AI-generated poetry and pseudo-criticism to think about what happens when statistical data about the proximity of tokens in an LLM evacuates the historical significance of words and phrases. We spend some time considering how these and other problems relate to interface design and how AI companies attempt to mitigate them, testing different chatbots against a model’s ‘soul document’. The next sessions are themed around pedagogy and education. Participants will be presented with suggestions for ways that generative AI may be potentially helpful in the classroom, followed by hands-on work with a tool designed to help communities of practice—such as secondary school teachers and university tutors—establish norms for acceptable AI use among themselves. We will think about the post-generative AI classroom in terms of designing for critical AI literacy, and look at examples of learning and assessment tasks which build in AI literacy without compromising the core content of the course. In the final sessions, we will think about changes to information ecologies and the authentication of knowledge from a range of disciplinary perspectives, including law, information science and literary theory. We will move to a critical interrogation of AI hype as articulated by tech companies, including its roots in science fiction, and will discuss insights from key thinkers from feminist STS such as Lucy Suchman and Donna Haraway on the necessity of understanding AI as something other than a singular, reified entities. Participants will be asked to read a number of short, accessible sources setting out the consequences – social, cognitive, psychological, democratic, economic, environmental and more – of AI and other algorithmically driven technologies whose emergent effects are gradually coming to light, and this will feed into group discussion. Sources will be gathered in an online group library which will act as a resource to which participants will continue to have access after the course is over, and to which they can add further sources if required. The course will conclude with an audit of where machine learning is incorporated into participants’ lives and how its effects might be mitigated.
Intended outcomes
At the end of the course, participants should have a sufficient grasp of the probabilistic functioning of large language models to be able to critically distance themselves from modes of reading which draw readers into interpreting LLM outputs as human-authored. They should be able to situate AI chatbot outputs within a longer historical trajectory of text generation technologies, and should have a grasp of some of the downstream effects and social harms these technologies are causing. They will have an appreciation of the effects of LLMs on some of the institutions important to a functioning democracy, and will be given resources to take away to help with the ongoing project of taking a critical stance towards generative AI in their research and teaching.
Prerequisites
Familiarity with the command line/bash shell: participants without this can get up to speed with the Shut Up and Shell tutorial at https://cglab.ca/~morin/teaching/1405/clcc/book/cli-crash-courseli3.html. Basic familiarity with Python would be useful, but is not essential.