Posts made by Matt Bury

Picture of Plugin developers
My understanding is that although the models are getting larger, they're not improving much. The last I heard was that they had increased the model size by an order of magnitude but only observed marginal improvements. I also read from some researchers, I can't remember who, that GPT LLMs had pretty much reached their theoretical limits.

Anyway, that is all theoretical & speculative as are predictions of "breakthroughs" in models being just around the corner. Personally, I'll believe it when I see it. Until then, how about we give poor students a break & stop falsely accusing them of plagiarism based on unreliable evidence?
Picture of Plugin developers

I wouldn't say abandon. As you say it's early days. Wouldn't it be better to wait until the AI detectors work within acceptable degrees of accuracy before implementing them on students? Much the same way we'd like our dentists to make sure a new procedure has a minimum probability of success before trying it out on us?
Average of ratings: Very cool (2)
Picture of Plugin developers

AFAIK, the whole issue of the efficacy of explicit form-focused directive & epistemic feedback is still contentious even though nobody has yet provided good evidence that it improves writing accuracy in subsequent writing, i.e. not just a redraft. See the many articles by John Truscott on this, e.g.Truscott, J. (2019). The Effectiveness of Error Correction: Why Do Meta-analytic Reviews Produce Such Different Answers? In Epoch making in English teaching and learning: A special monograph for celebration of ETA-ROC’s 25th anniversary (pp. 129–141). https://www.researchgate.net/publication/335106040_The_Effectiveness_of_Error_Correction_Why_Do_Meta-analytic_Reviews_Produce_Such_Different_Answers

I very much doubt that the effects of explicit form-focused directive & epistemic feedback could account for an effect size of d .7, so I suspect that something else is happening between the students receiving what are essentially GPT LLM generated "recasts" of their compositions. Is it that they get to read & cast an analytical eye over high-quality versions of their previously expressed ideas & arguments & that it encourages linguistic uptake in some way? I'm very curious about how they got to those large effect sizes!
Picture of Plugin developers

Small #OpenAccess 11-week study reports that students receiving detailed, specific feedback on their writing, generated by ChatGPT, made better improvements than an established automated feedback system but the students felt worse about it. https://www.tandfonline.com/doi/full/10.1080/09588221.2025.2454541?mi=5fx7dw#abstract

Abstract

The affordances of ChatGPT in language learning and teaching have gained increasing traction. While studies began to investigate the potential of ChatGPT as a feedback provider, little attention was given to ChatGPT’s potential impact on students’ writing performance and the ideal L2 writing self vis-à-vis the established automated writing evaluation systems (AWE). To address these gaps, a sequential explanatory mixed methods design was adopted. One hundred and fifty second-year university students from three writing classes in a Chinese public university were recruited and randomly divided into a ChatGPT group, an AWE group, and a control group. After an eleven-week intervention, the ANCOVA results showed that while the ChatGPT group scored significantly higher than the AWE group and the control group in post-writing performance as measured by their writing score, in terms of students’ ideal L2 writing self, the ChatGPT group performed significantly lower than the AWE group with a medium effect size. Qualitative analysis of students’ reflection papers revealed students’ (over)reliance on the tool and the accompanying loss of creativity and agency. Pedagogical implications as well as directions for future research are also discussed.

Average of ratings: Useful (1)
Picture of Plugin developers

It's official: GPT LLM generated texts are not copyrightable, i.e. They're public domain & would therefore be Open Educational Resources. However, additional content designed by humans to leverage the texts are copyrightable, i.e. our own work.

"V. Copyright and Artificial Intelligence, Part 2: Copyrightability

CONCLUSION
Based on the fundamental principles of copyright, the current state of fast-evolving
technology, and the information received in response to the NOI, the Copyright Office
concludes that existing legal doctrines are adequate and appropriate to resolve questions of
copyrightability. Copyright law has long adapted to new technology and can enable case-by-
case determinations as to whether AI-generated outputs reflect sufficient human contribution to
warrant copyright protection. As described above, in many circumstances these outputs will be
copyrightable in whole or in part—where AI is used as a tool, and where a human has been able
to determine the expressive elements they contain. Prompts alone, however, at this stage are
unlikely to satisfy those requirements. The Office continues to monitor technological and legal
developments to evaluate any need for a different approach.
The Office will provide ongoing assistance to the public on the copyrightability issues
related to generative AI, including by issuing additional registration guidance and updating the
relevant sections of the Compendium of U.S. Copyright Office Practices. In doing so, we will rely
on the comments received in response to the NOI, judicial developments, and other relevant
input."

Reference: United States Copyright Office. (2025). Copyright and Artificial Intelligence, Part 2: Copyrightability (p. 52). United States Copyright Office. https://www.copyright.gov/ai/

To me, it looks like GPT LLM generated texts are a good fit with Open Educational Resources (OER). However, the ideational content of the texts is unreliable, i.e. may contain false &/or irrelevant or misleading information. With expert propmting & review they can at least speed up the process of OER development.

What do you think?

Average of ratings: -