Are AI papers now written by AI?
Are AI papers now written by AI? Why even ask the question. Of course they are. Everybody does this. Everybody asks their favorite LLM to write their paper. Well no, not everybody. I still painfully type out every word by hand. Nevertheless, I realize I’m old and set in my ways and younger people may have fewer compunctions about letting an AI write for them.
I have observed a trend recently that makes me suspect extensive AI use in AI paper writing: I keep encountering papers that describe common knowledge in extreme detail and with high precision, while the parts of the paper that are new and innovative are cryptic and barely make any sense. Say a paper proposes a new type of transformer architecture that introduces a minor modification relative to the standard transformer. The paper will have several pages explaining in exquisite detail how scaled dot-product attention works, what the equations are that govern it, how it is implemented, etc., and similarly for other standard components of the modern AI toolkit that could be simply stated in 2–3 sentences alongside a citation of the relevant paper. And then, when we get to the new parts, the actual innovations, there will be three sentences covering them, the sentences will lack depth, and they will be entirely insufficient to understand or reproduce what was actually done.
When I first encountered this type of paper, I was rather confused. Why would anybody spend so much time writing about stuff everybody knows and then skimp out on the parts that matter? If you’ve invented a new method or model architecture, wouldn’t you want that part of your work to be clearly explained, so others can understand it and build on it? And then, recently, it dawned on me. If they’re writing the paper with AI, it doesn’t take much time at all to write all the boring stuff that everybody knows. The AI will bang it out in a few minutes, and it’ll be written well and it’ll be correct, and it’ll make the authors look like they did a meaningful amount of writing. And then the AI will fail on the new parts, the parts it has never seen; and since the authors can’t write either those parts of the paper will be incomprehensible.
I will say that these types of papers seem to come primarily from certain geographic regions that I won’t name here. Let’s just say they don’t speak English there. So maybe they can be excused. It’s not easy to write in a language that isn’t your own.1 At the same time, this is literally their job. They are scientists, and presumably they want recognition for their work. So maybe putting in a bit more effort would be appropriate.
I’m curious. Have you encountered these types of papers? Is this trend real? Or am I generalizing from a few anecdotes I have come across? Let me know in the comments.
As a non-native speaker who has spent countless hours studying the English language, I only have so much sympathy for this argument. The resources are available. Anybody who wants to learn writing in English can do so.
I started to notice that type of writing in my government courses this summer: over-explanation of the background and under-explanation of the point.
I find that the summaries can be on-point but lack any compelling articulation of the contribution. I hate writing the literature review as much as the next person, but slogging through the relevant “conversation” is the only way to explain how my innovation moves the frontier.