Mezuen egilea: Matt Bury

Moodle in English -> Lounge -> Why do we perceive LLM-generated texts as inauthentic or fake?

Matt Bury -
Plugin developers-ren irudia

Re: Criticisms of LLM-generated texts...

Many people have commented on a particular quality of LLM-generated texts, i.e. that they are bland, generic, &/or lack "personality" or "voice," which I agree with. But what is it specifically, concretely that we notice?

I think it's the tendency to use generic vocabulary choices, less discipline-specific vocabulary that practitioners within a discipline expect. In other words, they don't sound authentic because they don't use the same words that human practitioners do.

This is a result of how LLMs work, i.e. through predicting statistical probabilities of the next morpheme based on the previous morphemes (inculding the input prompts). Since LLMs are built from pretty much ALL human texts, not discipline specific, this skews the probabilties towards more generic, more widely used vocabulary that practitioners within disciplines wouldn't use.

LLM-generated texts are therefore perceived as "inauthentic" (fake) because they don't conform to practitioners' expectations, norms, conventions within specific discourse communities.

I don't think this is limted to vocabulary & phraseology either. I suspect this "generification" runs through all levels of linguistic analysis from words to the structures of whole texts.

What do you think? Is that a valid point?

Puntuazioen batez bestekoa: -
Plugin developers-ren irudia
I have an anecdote (anecdote =/= evidence!):

I moved to a small town at the same time as a colleague. We both used Google Maps on our mobiles to find our way around. However, me being the CogSci boffin, I knew that simply following the directions would mean I wouldn't learn how to get around without assistance, i.e. I wouldn't develop a mental map of the town so that I could navigate independently. My colleague just followed the directions.

After a year, I could easily take the quickest route from A to B & leave my phone in my pocket. He still had no idea where anything was. He couldn't even point in the general direction! (No, I'm not exaggerating either; it really was that bad!)
Puntuazioen batez bestekoa: Very cool (2)
Plugin developers-ren irudia

Perhaps some background would be helpful. So far the emerging research on LLM use, not just for exams & assignments, but also for studying & course participation, shows harmful effects across multiple dimensions, e.g. 

Abbas, M., Jam, F. A., & Khan, T. I. (2024). Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students. International Journal of Educational Technology in Higher Education, 21(1), 10. https://doi.org/10.1186/s41239-024-00444-7
Lehmann, M., Cornelius, P. B., & Sting, F. J. (2025). AI Meets the Classroom: When Do Large Language Models Harm Learning? (No. arXiv:2409.09047). arXiv. https://doi.org/10.48550/arXiv.2409.09047
Salih, S., Husain, O., Hamdan, M., Abdelsalam, S., Elshafie, H., & Motwakel, A. (2025). Transforming education with AI: A systematic review of ChatGPT’s role in learning, academic practices, and institutional adoption. Results in Engineering, 25, 103837. https://doi.org/10.1016/j.rineng.2024.103837
Wecks, J. O., Voshaar, J., Plate, B. J., & Zimmermann, J. (2024). Generative AI Usage and Exam Performance (No. arXiv:2404.19699). arXiv. https://doi.org/10.48550/arXiv.2404.19699
Zhai, C., Wibowo, S., & Li, L. D. (2024). The effects of over-reliance on AI dialogue systems on students’ cognitive abilities: A systematic review. Smart Learning Environments, 11(1), 28. https://doi.org/10.1186/s40561-024-00316-7

Plugin developers-ren irudia
Can we please get back to the issue of helping teachers & institutions take some measures to address LLM-enabled academic misconduct?

What can Moodle LMS do to help?
Plugin developers-ren irudia

Re: "That is stone age. Do you claim that they apply equally to the AI age? What are your arguments?"

B.F. Skinner is credited with providing the theoretical basis for language labs in second & foreign language education (even though Chomsky tore this apart when he critiqued Skinner's book, "Verbal Behavior" (Chomsky, 1967)). The language lab pedagogical model is essentially what most language learning apps & now LLM interactive interfaces have adopted. So yes, it's pretty relevant.

The thing that's wrong with language labs isn't the technology that's being applied but the pedagogical model. The providers/designers of these new LLM-based language labs can't even specify their learning model or the underlying cognitive mechanisms by which they work. To quote Sweller, Ayres, & Kalyuga (2011), 

"Without knowledge of human cognitive processes, instructional design is blind. In the absence of an appropriate framework to suggest instructional techniques, we are likely to have difficulty explaining why instructional procedures do or do not work. Lacking knowledge of human cognition, we would be left with no overarching structure linking disparate instructional processes and guiding procedures. Unless we can appeal to the manner in which human cognitive structures are organised, known as human cognitive architecture, a rational justification for recommending one instructional procedure over another is unlikely to be available. At best, we would be restricted to using narrow, empirical grounds indicating that particular procedures seem to work. We could say instructional procedure A seems better than procedure B but why it works, the conditions under which it works or how we can make it work even better would be rendered unanswerable and mysterious."

Since all education, including STEM, revolves around language, that LLMs have no underlying schemas or models of knowledge or learning, what they generate is superficially convincing, giving the appearance of "good answers" but loaded with misconceptions, over-generalisations, inappropriate conflations, & downright pseudoscience. Essentially, LLMs have no principles or integrity in any sense of the word. This may get overlooked in political, PR, & marketing circles but in scholarly contexts it becomes apparent very quickly.

I suspect this is one of the causes of frustration in the students who complained about their LLM generated course materials. They really are a waste of time if they haven't been thoroughly audited & "made meaningful, valid, & reliable" by subject matter & instructional design experts, which typically requires a lot of work & it's sometimes more efficient for experts & IDs to just write it themselves.

References
  • Chomsky, N. (1967). Review of B. F. Skinner’s Verbal Behavior. In L. Jakobovits & M. Miron (Eds), Readings in the Psychology of Language (pp. 142–143). Prentice-Hall. https://chomsky.info/1967____/
  • Skinner, B. F. (1957). Verbal Behavior. Copley.
  • Sweller, J., Ayres, P. L., & Kalyuga, S. (2011). Cognitive Load Theory. Springer. https://www.springer.com/us/book/9781441981257
Puntuazioen batez bestekoa: Very cool (1)