Teaching data visualization in the time of generative AI
If students want to cheat with AI, the least I can do is make them own up to it.
There are dark clouds on the horizon for my class on data visualization. I enjoy teaching this topic, but the entire setup for my class is being upended by generative AI. I no longer know how to create meaningful assignments. How can I assess whether students have learned anything when they can complete entire data analysis projects with a quick request to ChatGPT?
My class is about hands-on working with and visualizing data. It inherently requires that students work on assignments at a scale of multi-day projects. Students need to familiarize themselves with a dataset, perform some exploratory investigations, create visualizations, and then document what they have done and what they can conclude from their analysis. This type of work can only be carried out in the format of a take-home assignment. And it is also something that ChatGPT or Claude Sonnet could crank out in minutes in response to a simple prompt.1
In reaction to the rise of generative AI, some teachers have transitioned their assessments to either in-class exams written by hand or oral assessments conducted in person. Neither approach works for me. The main learning outcome for my class is the ability to produce a compelling report that combines written text, computer code, and visualizations of the dataset analyzed. This skill cannot be tested with a hand-written, hour-long exam. An oral exam could make sense—I could ask students to walk me through their analysis and explain step by step what they did and why—but oral exams don’t scale. I have a hundred students in my class. I cannot possibly examine every single one of them in person.
Until recently, I felt like giving up. The only way forward I saw was to ignore the issue, teach the class as I always have, and let the dice fall where they may. Ideally, students would work on their assignments with minimal reliance on generative AI, because I would ask them to do just that. But of course I would not have any means of enforcing this behavior. And I don’t believe in AI detectors. The last thing I want to do is argue with students over whether or not they have employed AI to complete a given assignment.
However, last week I had an idea that hopefully will make the AI situation a bit more bearable. At a minimum, it will require the students to reflect on their AI use. Starting next spring,2 I will ask the students to declare, at the end of each assignment, how they used AI models in the preparation of their work. This idea is inspired by similar policies now put in place by machine learning conferences, see for example here for NeurIPS. I can’t prevent students from turning off their brain and letting the AI do all the “thinking,” but I can make them document what they did. This should prompt them to reflect on how they’re engaging with AI. And because in my class students peer-grade each others’ assignments, I hope it may make them feel uncomfortable having to admit to their peers that their entire assignment was written by AI. Maybe this will convince them to use AI a little less.
Of course all of this is based on the honor system. A student who intends to lie and deliberately obscure that their assignment was AI generated has that option. There’s not much I can do about that. But then, even without AI, students can ask their best friend or brother or hired help to write their essays and pretend it was their own work. Students who want to cheat will always find a way. There’s little I can do to prevent every such possibility. I see my job as teaching to the students that actually want to learn. My primary goal is to create a learning environment where students are motivated and inspired to make an honest effort. I hope this new AI policy will help me achieve this goal.
I don’t actually think these AI-generated analyses and visualizations are very good, but they’re usually good enough to get a decent grade in my class. At a minimum, they tend to be better than what the weakest students in the class produce.
That’s when I’ll teach the class again.
Hm, yeah, this seems like a real challenge...
In addition, maybe it could be interesting to turn this into an AI study, where students compare different AI models and making this a benchmark? I.e., which model was the easiest to work with to complete the assignment most satisfactorily? How many and what types of interventions were needed, etc.
I think AI has exposed the fundamental difficulty all education has faced forever: how do you get people to learn things they don't want to learn? Obviously making them want to learn would be superior, but if that fails, what can you do?
Now, with AI, it really seems like the answer is, "nothing". I vaguely support capping the highest possible scores on all standardized tests - because as is, people end up putting in ridiculous amounts of effort into their grades or SAT scores, effort that teaches them that school is all about getting the grade, effort that teaches them that learning about things that are interesting and useful and practical is a waste of time and that rote memorization and brute force pattern recognition is all that matters. It's Goodhart's Law maxed out to the extreme: how is it even possible that people can make functional businesses on SAT prep courses and classes?
Yes, you have to force children to get a baseline of education. Basic math, basic literacy, basic civics and history. You should also give them ample opportunity to develop their interests, maybe involving forcing them to take like one class on each of the arts or something, and connect them with things that actually have an impact in the real world. But beyond that, more effort should be taken to ensure that the kids actually want to do things - otherwise you increase the number of adults who are high-achieving robots.