Working Group Roadmap Forum

Analytics in the news

Picture of Elizabeth Dalton
Analytics in the news

Here's a thread where any of us can post news items or articles we've seen about analytics and machine learning.

My first contribution: 

Unintended Consequences and Goodhart’s Law: The importance of using the right metrics

"Goodhart’s Law is expressed simply as: “When a measure becomes a target, it ceases to be a good measure.” In other words, when we set one specific goal, people will tend to optimize for that objective regardless of the consequences. "

Let's think about how this could affect learning analytics. If we don't choose our targets carefully, participants may optimize their participation too narrowly. Here's a example given in the article:

"In school, we are given one objective: maximize our grade. This focus on one number can be detrimental to actual learning. High school seemed like one long series of memorizing content for a test, then promptly forgetting it all so I could stuff my brain full of info for the next one, without any consideration of whether I really knew the concepts. This strategy worked quite well given how success was measured in school, but I doubt it is the best approach for a great education."

One of the significant-- and often overlooked-- advantages of learning analytics is their potential to detect hidden or "latent" factors in the learning process, not just predict a learning product. What kinds of processes could we detect that would be less prone to "gaming the system" than a final grade prediction?

Picture of Cristian T. Duque
Re: Analytics in the news

Not sure if I am allowed to make contributions, but this caught my eye:

What Parents and Teachers Think About Education Data (September 12, 2018)

Personally I am positively surprised at the reported open-mindedness of parents regarding use of data about their children by teachers.

On the teacher level, the challenges are reasonable and known, at least I assume as much for members of this community: limited skills, limited time, low prioritization by officials, insufficient development of supporting tools.

It also links to a 4-step policy plan to "Make Data work for Students" which at first glance I believe it is compatible with Moodle Learning Analytics approach

Full disclosure, I do not know more about the Data Quality Campaign, its motives, nor its potential conflicts of interests than what shows on its website. It is largely funded by philanthropies from wealthy people and corporations.