Paper review: Beyond Clickometry

Paper review: Beyond Clickometry

by Michael de Raadt -
Number of replies: 16

Beyond Clickometry:  Analytics for Constructivist Pedagogies

Picture of Aleksander Dietrichson

Author Aleksander Dietrichson
Publication International Journal on E-Learning
Date 2013
Link http://www.editlib.org/p/38478/
File
 Paper link, Details

Learning analytics has become a topic of much fundamental research in recent years. Researchers and educators are coming to terms with the collective data that is available to them and determining what can be done with such data.

In his paper, titled "Beyond Clickometry:  Analytics for Constructivist Pedagogies", Aleksander Dietrichson explores the data available from an LMS (Learning Management System).

The key point of the paper is that past studies have relied heavily on behavioural log data, but by  estimation, this constitutes only 15% of the relevant data available from an LMS, such as Moodle; the other 85% being constituted of unstructured data that represents user content.

"Only about 15% of all data stored is stored in a structured format allowing for easy analysis. The remaining 85% is unstructured data and often go un-analyzed."

In order to achieve a constructivist pedagogy, the analysis of the interactions among students needs to be "enhanced" by the inclusion of both behavioural and unstructured data, according to Dietrichson, who provides evidence from a case study to back this up.

"While sophisticated clickometry certainly has a place in the realm on LMS analytics, it needs to take its place alongside methodologies more suited to evaluating the pedagogical processes practitioners aspire to facilitate in their on-line courses."

Average of ratings: Useful (1)
In reply to Michael de Raadt

Re: Paper review: Beyond Clickometry

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Rather annoying of you to review a paper that is not open access. It is not even available through the OU library sad

A paper that currently freely available, and which discusses the type of detailed analytic one can do on quiz data is, http://oro.open.ac.uk/40579/2/e-assesstolearn_2013.pdf

Average of ratings: Useful (2)
In reply to Tim Hunt

Re: Paper review: Beyond Clickometry

by dawn alderson -

Morning/Afternoon both

Well, I suppose this reminds me of the age-old issue surrounding the value of formative assessment, in terms of overall outcomes and related perceptions (both teacher and learner) to do with summative assess.

I am not necessarily talking about teaching to/learning for 'the test', but I guess if teachers/learners are to realise the actual value of data analytics for practice, such measures/outcomes need to be embedded in a context with a clear purpose, that is, in order to make sense to those involved i.e. the teacher and the learner smile

As it stands, and my view is informed by those papers...Dietrichson makes an outstandingly, superb point when he states:

"Only about 15% of all data stored is stored in a structured format allowing for easy analysis. The remaining 85% is unstructured data and often go un-analyzed."

Closing that gap, it seems to me, sounds like a plan.

D    

Average of ratings: -
In reply to dawn alderson

Re: Paper review: Beyond Clickometry

by aleksander dietrichson -

Indeed, the gap needs to be closed -and it is important to realize that this is a research problem, not a technological one, i.e. it is the methodologies used to analyze the data is what turns it into useful information. 

Average of ratings: -
In reply to aleksander dietrichson

Re: Paper review: Beyond Clickometry

by Sean Dicks -

Hi.  I am a psychologist, and I will be doing research to understand how a facilitator, from a psychological point of view, can contribute to an  increase in trust, relationships and interaction when engaging with a group online.  I will be looking at the "behaviours" of the facilitator, and would also need to develop measures of the ways that those specific behaviours influence groups. 

I found your article very relevant, and I agree that technology is able to race forward in leaps and bounds, and it is up to us humans to determine the ways that that technology can be best used, and how we should position ourselves in our relationship with technology to obtain the most benefit.   We live in a world where we constantly have new tools available.  We need to make the time to research and understand those tools so that we use them well and gain insight that ensures that the next generation of tools builds on those we have today.  

Is there a specific journal that I could consult, or website that I can visit to find out more about your "x-ray research"?   Thanks for a great article - Regards  Sean

 

Average of ratings: -
In reply to Tim Hunt

Re: Paper review: Beyond Clickometry

by Sean Dicks -

This is also an interesting article. (Jordan 2014)   The time that students have available is commented on as a contributing factor to the different performance of the two groups.  It would be interesting to explore the ways that age differences in the two groups played a role, and whether future group A students would benefit if this article was presented to them at the beginning of their course.  If they can "see" their future behaviours, and the consequences thereof, will they change them?

Average of ratings: -
In reply to Michael de Raadt

Re: Paper review: Beyond Clickometry

by Michael de Raadt -

Aleksander has kindly allowed us to share the paper. I will add a link to the original post in this discussion.

Average of ratings: Useful (1)
In reply to Michael de Raadt

Re: Paper review: Beyond Clickometry

by dawn alderson -

Hi,

Thanks for this, interesting. A number of points then.

1. The ultimate point of the paper, and I paraphrase:

...more qual methods need to be applied to LMS data

2. Notion of clickometry-nice concept, new concept and the author provides solid arguments.

3. However, relying on ratios for  analysis, well, it just doesn't sound strong to me.  

4. Social learning support to inform the sociogram outcomes is tried and tested over, and was nice to see in the paper-doesn't go far enough though.

5. More research about interaction is an absolute must. 

I enjoyed reading the paper.

D


Average of ratings: -
In reply to dawn alderson

Re: Paper review: Beyond Clickometry

by Sean Dicks -

I found it interesting that the depth of information about interaction as well as learning and constructing knowledge (hypothetically linked to use of words) could so easily be extended with reference to readily available data.

Using the same system, it would probably be possible to add a dimension of course topic as an area of analysis - is there an increase in discussion around a specific part of the course, or at a specific time? 

In order to improve the computer's ability to understand the categories that information (such as words) can be placed into, a database can be built that facilitates the removal of some words from analysis (e.g. "the") while giving weight to other words chosen by the course organiser.

I am considering conducting my PhD Psychology using online forums and potentially the Moodle environment to investigate factors such as trust, interaction and relationship-building in an online environment.  I am particularly interested in the actions that can be taken by the facilitator in that environment to enhance interaction, relationships, and knowledge -building.

In my Masters training the emphasis was on a constructivist epistemology, and the social constructionist view of the Moodle environment fits well with my ideas.

Sean

Average of ratings: -
In reply to Sean Dicks

Re: Paper review: Beyond Clickometry

by Sean Dicks -

Its interesting that Dietrichson (2013) works in x-ray research.  This must be useful when considering how to "see through" the obvious clickometry data.

Average of ratings: -
In reply to Sean Dicks

Re: Paper review: Beyond Clickometry

by dawn alderson -

Well, good morning Dicks!   Sorry, I mean Sean evil

Sean, you can always email the author for further info-as I have done, not for info, I just let him know about his solid arguments there in the paper.

On the note of social constructionist env....a good way to remember it, I always find, is by thinking about a group of chimps.  Tis true, I once saw an Attenborough programme.........approximately 7 chimps stood in a circle....and took it in turns to defecate in the middle....then stand back and do a sort of merry dance in watching over their shared artefacts.......yep-that is a good way, well a humourous way to remember SC...which is what I used to tell my students too smile

cheers all (LOLs!¬!!!!)

D 

Average of ratings: -
In reply to dawn alderson

Re: Paper review: Beyond Clickometry

by Sean Dicks -

Hi

I watched a movie not so long ago titled "Dawn of the Planet of the Apes", though I am not entirely sure why I thought of that now  smile

Anyway, that is a brilliant analogy, especially given that the ground would have become fertile, allowing the growth of new plants (which acts as a metaphor for the the growth of new ideas from the offerings of the group)

 smile

Average of ratings: Useful (1)
In reply to Sean Dicks

Re: Paper review: Beyond Clickometry

by dawn alderson -

oh please, don't! that is the exact movie I set out to watch on a flight recently......only to find that the contents of the bar got in the way and as a consequence I couldn't ascertain how the media equipment worked...........blush 

Anyway, Sean you are right when you say this:

Anyway, that is a brilliant analogy, especially given that the ground would have become fertile, allowing the growth of new plants (which acts as a metaphor for the the growth of new ideas from the offerings of the group)

Been thinking about sociometric data, sociograms etc as a qual data collection tool-prompted by this thread. It was Robin Banerjee at Sussex who came up with a data collection tool that sought to look at interpersonal relations across school kids, here is the link:

http://www.sussex.ac.uk/Users/robinb/socio.html

I think, something like this with fancy algorithms that would enable data collection about learning engagement/interaction/motivation indicators and so on would be a great tool for online L&T.

Cheers,

Dawn 


Average of ratings: Useful (1)
In reply to dawn alderson

Re: Paper review: Beyond Clickometry

by Sean Dicks -

Well, I don't think the algorithms would need to be complex.  Data about who responds to who, and who "avoids" who is readily available from a discussion thread as described in the Clickometry article. 

I guess this information can be put in context using a simple survey of the attitudes of the class members to each other, and their learning styles. It is possible that someone may not be too involved in interaction simply because they prefer to learn on their own.  They may be self-motivated and successful in some sense, although they rank low on interaction.

Having such info would be great, but I would be concerned that some facilitators may rely on that data (especially as the computer gets better at understanding it) rather than actually interacting or seeking to understand the class in a more direct personal way.  It would be important to remind ourselves that the data should prompt action or intervention as appropriate. 

Robin here identifies the orange class members as representing good leverage points.

Average of ratings: -
In reply to Sean Dicks

Re: Paper review: Beyond Clickometry

by aleksander dietrichson -

"Its interesting that Dietrichson (2013) works in x-ray research."

Not the first time I have been mistaken for a radiologist. wink

X-ray research is actually a learning analytics company, the metaphor, of course, being that the analytics platform can help you see what goes on below the surface.

Average of ratings: Useful (1)
In reply to aleksander dietrichson

Re: Paper review: Beyond Clickometry

by dawn alderson -

hi, found this neat little resource:

https://net.educause.edu/ir/library/pdf/ELIB1101.pdf

dates back to 2011. and nice to see mention of dialogue- based activity, specifically Neil Mercer's work at the OU-I think, last I knew he was at Cambridge... 

So, basically LA...learning analytics is the thrid wave...and can only get better because of web 2. But, I am thinking we are soon to reach 2015, and thus 4 years on...it is a shame we have not moved further in respect of metrics and what not...it just makes me think about my previous thoughts in that Web 3 might bring about a broader range of possibilities for learner-analytics and all that joined up nice stuff Cath spoke about.

Anyway, tis worth a read if you have not come across it before.

D 

  

Average of ratings: -