Artificial Writing and Automated Detection [pdf]
14 comments
·October 27, 2025binarymax
nonethewiser
Do teachers prefer grading papers or something? This always seemed like the obvious answer and there are no shortage of complaints. There is something making papers "sticky" that I do not understand. Education needs to be agile enough to change it's assessment methods. It's getting to the point where we can't just blame LLMs anymore. Figure out how to asses learning outcomes instead of just insisting on methods that you assumed should work.
binarymax
Because, assuming it's done properly w/o cheating, it's a great learning tool. It's sometimes easy to forget that certain tasks are the way they are because they're supposed to teach. We don't structure teaching and learning around what the least painful thing is.
NewsaHackO
I think the most realistic way is to do a flipped classroom, where middle-school and beyond, children are expected to be independent learners. Class time should be spent on application of skills and evaluation.
laptopdev
If computer usage hampers a child's socialization with the group he's learning with, maybe the simplest and most meaningful solution would be preventing children enrolled in language comprehension classes from having access to computers at home particularly at core language and reasoning stages in development.
Legend2440
I suspect AI text detection has actually become easier, as chatbots today have been heavily finetuned towards a more distinctive style.
For example “delve” and the em-dash are both a result of the finetuning dataset, not the base LLM.
AuthAuth
You are forgetting the human mind accounting for this and adding "write this like a kinda dumb high school student". I just did a little test between a copilot essay and the same prompt with "write this like a kinda dumb high school student" and it reads like an essay i would have written.
haffi112
That's where the humanizers come in. These are solutions that take LLM generated text and make it sound human written to avoid detection.
The principle of training them is quite simple. Take an LLM and reward it for revising text so that it doesn't get detected. Reinforcement learning takes care of the rest for you.
andy99
While it’s interesting work, so far my experience is that AI isn’t good enough (or most people aren’t good enough with AI) for detection to really be a concern, at least in “research” or any writing over a few sentences.
If you think about the 2x2 of “Good” vs “By AI”, you only really care about the case when something it good work that an AI did, and then only when catching cheaters, as opposed to deriving some utility.
If it’s bad, who cares if it’s AI or not, and most AI is pretty obvious thoughtless slop, and most people that use it aren’t paying attention to mask that, so I guess what I’m saying is for most cases one could just set a quality bar and see if the work passes.
I think maybe a difference AI brings is that in many cases people don’t really know how to understand or judge the quality of what they are reading, or are to lazy to, so have substituted as proxies for quality the same structural cues that AI now uses. So if you’re used to saying “it’s well formatted, lots of bulleted lists, no spelling mistakes, good use of adjectives, must be good”, now you have to actually read it and think about it to know.
vages
I personally would value a spam filter that filters out AI generated content.
rawgabbit
Wow. Never heard of Pangram until now. Quote:
Pangram maintains near-perfect accuracy across long and medium length texts. It achieves very low error rates even on shorter passages and ‘stubs.’alfalfasprout
I'm extremely skeptical of these claims. Especially when we're dealing with careful prompting to adjust tone/style.
haffi112
Even if it was close to being near perfect, that is still not enough due to the negative impact of false positive detections on students.
zingababba
Mmmm yes, I probably will never be able to find it again but someone recently tested a lot of these out and found you could bypass them easily by changing a few words around.
My two cents about this after working with some teachers: this is a cat and mouse game and you're wasting your time trying to catch students writing essays on their own time.
It is better to pivot and not care about the actual content of the essay, but instead seek alternate strategies to encourage learning - such as an oral presentation or a quiz on the knowledge. In the laziest case, just only accept hand-written output - because even if it was generated at least they retained some knowledge by copying it.