Paper review: Three paths to learning analytics

Paper review: Three paths to learning analytics

by Michael de Raadt -
Number of replies: 7

Three paths for learning analytics and beyond: moving from rhetoric to reality

Picture of David Jones Picture of Rolley Tickner Picture of Colin Beer

Authors Colin Beer (CQU), Rolley Tickner (CQU), David Jones (USQ)
Publication Proceedings of ASCILITE2014
Date 2014
Link Paper link, Details


Learning analytics (LA) is a topical area of educational technology research. Many are sure of the potential of learning analytics to improve online and blended learning outcomes. There are a number of small-scale systems being trialled around the world, but the promised benefits of LA are yet to be fully realised, according to this paper presented at ASCILITE in Otago, New Zealand late in November 2014.

Just Over the Horizon

The paper begins by recognising the rhetoric around LA and attributes it to managerial decision making processes, which the authors say are often driven by fads.

Will learning analytics be a repeat of past mistakes where technology implementations fail to move beyond a transitory fad and provide meaningful and sustained contributions to learning and teaching?
As evidence, Beer et al note that LA has had some mention in the Horizon Report since 2010. In 2012 LA was declared to be "one year or less" away. Again, in 2014, LA was "one year or less" away. However, in 2014, LA was considered "two to three years" away.

The authors note that adoption requires resolving technical barriers as well as encouraging use.

Three Paths

Beer et al provide three possible ways for "how LA can avoid becoming yet another fad".

  • Do it to teachers, where institutional change alters the teaching context.
  • Do it for teachers, driven by an alliance of innovative teachers, ICT and vendors.
  • Do it with teachers to determine how LA can help in the teaching context.

The authors recommend a balance of the three approaches, including "do it with" as a must for LA projects.

Beer et al back up their view through literature and also go on to describe EASI, an intervention system working towards greater engagement, currently in initial use.

This paper is attached to this post and is worth a read for all researchers and developers working towards LA tools right now. You can also access the slides used in the presentation of this paper at ASCILITE2014.

Average of ratings: Useful (1)
In reply to Michael de Raadt

Re: Paper review: Three paths to learning analytics

by Sean Dicks -

This is an interesting article.  I can see how taking a wider view of the system, and including the relationships between teachers, the technology, the university all working WITH each other produces good results.  I would make two suggestions.  Firstly, the model seems to be doing things to, or for the students.  Taking Beer & Jones' model further, undertstanding what motivates or demotivates students, and inviting them into the "black box" could extend the system to one that also Does it With students.  (Diagram 1)

Secondly, the use of Trigwell's model (p245) seems somewhat incongruent with the earlier statement that in complex systems part are all "interacting and evolving so that we cannot understand any of the agents or systems without reference to the others".  If the eco-systemic model was applied to the elements of Trigwell's model, the system should look like my diagram 2, where all connecting lines indicate relationships of mutual influence.  Trigwell's model seems linear rather than circular.

Attachment diagram 2.jpg
Attachment Diagram1.jpg
Average of ratings: Useful (1)
In reply to Sean Dicks

Re: Paper review: Three paths to learning analytics

by Sean Dicks -

Hi    The diagrams got switched around.  The second one is diagram 1

Average of ratings: -
In reply to Sean Dicks

Re: Paper review: Three paths to learning analytics

by David Jones -

G'day All,

Thanks to Michael for reading the paper and sharing it.  Thanks to Sean also for sharing some thoughts on the paper. I thought I'd share some explanation of the background of the paper which influenced the thrust of the paper.

In short, the focus on teachers rather than students in the paper comes from our increasing disquiet with the rise of the "do it to" approach around educational technology within Australian higher education (this is the context we work within). This is especially obvious within learning analytics over recent years.

With this path you have senior (and not so senior) management responding to the fad, identifying a tool and implementing it with the expectation that it will improve learning and teaching. It generally doesn't provide any significant, wide-spread change.

We don't disagree that the ultimate focus is on "do it with" the students. The question is how best do you achieve that goal within higher education as it currently stands?

And that's where Trigwell's model enters the picture. i.e. the idea that what the student experiences is largely (but not completely) influenced by the strategies, planning and thinking of teachers.

Trigwell's model of teaching


Our concern is that the current focus on the "do it to" path attempts to ignore this and instead believe that it's possible for the institution to directly impact the student experience. Whereas the Trigwell model (or at least one interpretation of it) suggests that the best the "do it to" approach can achieve is to change the teaching and learning context.

The "do it with" approach is focused much more on engaging directly with teachers' strategies as the starting place. Almost like desire paths, build on what they are already doing.  In part, the assumption is that people learn from experience, change their experience (strategies) and you have a better chance of changing their thinking.

Actually, the following slide (click on it to see a larger version) from an earlier presentation that contributed to the paper visually represents some of this and also touches on the complexity issue.

Slide65


In terms of complexity, my thought would be to stick with the Trigwell model, but to understand that while (at best) the teaching and learning context may be the same each of the subsequent layers will be unique to each individual teacher and student. i.e. each teacher will think differently, that will in turn translate into different approaches to planning and hence strategies. In addition, each student will respond to those strategies in different ways.

Hence the multitude of arrows in the above and the double headed nature of the arrows. Double headed is meant to suggest an inter-dependence. If you change a teacher's strategy (and thus their experience), you will hopefully change their planning, which in turn will change their future strategies, which in turn will...

I think the overlay of the double-headed arrows helps bring our interpretation of the Trigwell model closer to your second diagram (the one that showed up first in your post).  Correct?

David.

Average of ratings: -
In reply to David Jones

Re: Paper review: Three paths to learning analytics

by dawn alderson -

Thanks.

For now, I find this most interesting:

In short, the focus on teachers rather than students in the paper comes from our increasing disquiet with the rise of the "do it to" approach around educational technology within Australian higher education (this is the context we work within). This is especially obvious within learning analytics over recent years.

Really?  I am not aware of any evidence....be great to know where to look..Journal-wise.

D

Average of ratings: -
In reply to dawn alderson

Re: Paper review: Three paths to learning analytics

by David Jones -

G'day Dawn,

Sorry, I don't have any pointers to journal papers where this is discussed specifically in relation to educational technology. However, this tendency does get some mention in discussions around the rise of managerialisation within higher education. A quick search of my Mendeley collection reveals Barry et al (2001) as covering some of this ground. I'm sure it wouldn't be too hard to find similar and more recent discussions.

Actually, doing another quick search for "neo-liberal" another related label reveals Greenwood (2012) that covers some of this ground.

Disclaimer: this is the result of a quick, random search.

The position in our paper comes from our experiences during our careers at two Australian universities over 20+years. It's been supplemented by conversations with similar folk at other universities.

Our paper is partially an attempt to show that requirements of the "do it to/for" approach are built into the common project-management practices of institutions that pay little or no notice of "do it with". The story told in the paper seeks to illustrate how one apparently successful learning analytics project was only possible due to a large amount of "do it with" that was done outside the confines of traditional project-management practices.

Another paper from ASCILITE that I co-authored offers a couple of other illustrations That paper suggests that there is a specific mindset that is mostly implicit in standard institutional e-learning practice that is creating problems and suggests one alternative.

David.

Barry, J., Chandler, J., & Clark, H. (2001). Between the ivory tower and the academic assembly line*. Journal of Management Studies, (January). Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/1467-6486.00229/abstract

Average of ratings: Useful (1)
In reply to David Jones

Re: Paper review: Three paths to learning analytics

by Sean Dicks -

Hi David

Thank you for your reply, comments, and explanation.  I agree that the slide that you included demonstrates the complex dynamics within a system, and is closer to my diagram.  (I am still concerned though that the student's role may be depicted as being more passive than it needs to be)

I also agree that the influence is on the context that the other elements interact with, rather than directly on those elements themselves.   For example, in a hypothetical school, a teacher may paint the classroom blue, and then observe that students seem more calm.  It would not be that the teacher has made the students calm, but that the students are responding to a new context.

If the teacher engaged in self-reflection, they may realise that they also are more calm, perhaps linked to the interaction observed between the students and the "calm context".  With the teacher being calmer, the students may become even calmer, and so on.  In this way, all elements of the system influence and are influenced by others.

Perhaps I should mention that I am not a teacher / lecturer, and therefore I am commenting as an "outsider".  As such, I don't claim to understand or have any answers, and I would see myself as being curious, rather than knowledgeable on the topic.  Regards  Sean


Average of ratings: -
In reply to Sean Dicks

Re: Paper review: Three paths to learning analytics

by dawn alderson -

I see. 


OK, I think this thread is interesting...I want to read the resources and think more about it.  Although I must say...at this point, that is, not having read through the links thoroughly...I always think head-brick wall-bang! with this topic.  However, I too am curious Sean...and don't you be off anywhere.....your contributions are valid...lecturer or not eh!

D   

Average of ratings: -