Paper review: Creepy Analytics and Learner Data Rights

Paper review: Creepy Analytics and Learner Data Rights

by Michael de Raadt -
Number of replies: 9

Creepy Analytics and Learner Data Rights

Picture of Kay Souter Picture of Carolyn Woodley Picture of Scott Beattie

Authors Scott Beattie (CQU), Carolyn Woodley (RMIT), Kay Souter (Deakin University)
Publication Proceedings of ASCILITE2014
Date 2014
Links Paper link, Details


With the growing use of user data for learning analytics, it is worth taking a step back to envisage where such practices may lead. This position paper, presented late last year, attempts to provide a perspective of analytics that seems uncomfortable to consider now.

The paper begins by imagining a sales pitch of a company offering a multitude of data, gathered within and beyond the LMS. It paints a dystopic picture of what collecting information about individuals could lead to. The paper goes on to justify concerns raised (and also to justify the use of the word "creepy" in an academic paper).

The authors ask who owns student data and who should have access to it. They argue that collection of information for its own sake creates a real chance of identity theft. Where such collections are commercial organisations, it could be possible to misuse learner data for commercial gain.

Reaching a less creep future

The paper refers to a number of previous works that have suggested ways of protecting student data used for analytics.

To earn the trust and encourage the engagement of students, learner data systems need to be open (Siemens et al, 2011) rather than based on proprietary technologies, transparent, personalised, networked (Dawson et al, 2011), transportable (Buckingham-Shum & Ferguson, 2012), adaptive and interactive.

The authors suggest that there needs to be a focus on learner-centred learning analytics. This view could help us to not only protect student data, but also to focus on storing only the particular data we need to provide analytics, rather than simply collecting lots of information while we consider what to do with it.

They suggest working towards a charter, based on current legislation, that ensures student rights and encourages appropriate behaviour by educational institutions and ed-tech providers. There are also a number of other guidelines suggested to ensure data protection and privacy.

The paper, attached to this post, was presented at ASCILITE2014 and the presentation used is available online.

Discussion Questions

  • Is this a realistic view of the future? Is it possible or just FUD?
  • Who owns student data and who should have access to it? Does it belong to the student or the institution that collects it?
  • What do we have to do to ensure the benefits that analytics can bring while avoiding the problems?
Average of ratings: Useful (1)
In reply to Michael de Raadt

Re: Paper review: Creepy Analytics and Learner Data Rights

by dawn alderson -

Hi, I am yet to read the paper...looking forward to it and will return with more thoughts.

1. The authors suggest that there needs to be a focus on learner-centred learning analytics: Is this a realistic view of the future? Is it possible?  Yes, I think elements of a shared approach in terms of learner analytics-so with the learner, already takes place, but it is not systemised in an LMS, as far as I am aware, I could be wrong-but I am talking about for example, performance position in cohort, needs analysis on an individual basis in terms of stretching the student, hot-spots for development and so on.  

There is obviously evidence of tools that can support this in Moodle, in that students can view certain achievement/attainment outcomes in addition to face to face progress meetings with staff.  It is probable that Moodle could develop further to support a wider approach for shared LA with the learner, which  includes progress charts for those items and related items such as student diagnostics, development of reflective activity/metacognition-other skill-sets, knowledge and understanding for formative and summative outcomes. How? Streamlined gradebook access for the learner that has in-built mini systems that the student can access might be a good starting place-perhaps.

2. Who owns student data and who should have access to it? Does it belong to the student or the institution that collects it?   I want to think more about this.

3. What do we have to do to ensure the benefits that analytics can bring while avoiding the problems? A  focus on 1 as well as liaison with student union to gauge a view of 'student need' amidst other strategies perhaps.  Student voice about the development of LA is essential. 

D

Average of ratings: -
In reply to Michael de Raadt

Re: Paper review: Creepy Analytics and Learner Data Rights

by ryan sanders -
such broad range ya gave there, in the 4 different links. and then re-focusing discussion questions...  


realistic... Siemens et al, 2011  and  Buckingham-Shum & Ferguson, 2012 and 69-Beattie.pdf i would say yes as an overall. with ability to say no, in more finer niddy griddy detail.

realistic.... presentation used is available online. more of a SHOCK and AWE.  

===================

who owns student data and who should have access to it? does it belong to the student or the instution that collects it?

i don't want to touch that, even if had unlimited funds for lawyers.  with that said,....  it would be nice to have something like a GPL / open source license, of what your information can be accessed freely.  and then what other information must be classified...   username/passwords. = keep folks from changing your stuff exception for example *picking on forum moderators (waves to Michael de Raadt)*. but not state who can access it and do something with it.  

i would think if i went into moodle and field out say facebook information in profile field, i would be implying the moodle site can get information from my facebook page and other information from facebook. 

facebook is littered across many websites "sign on with facebook" and different website software ask for different information, and displays that information, before accepting the signin conditions. (this website will get your email address, billing address, shipping address, and also able to post to your facebook page, etc..). BUT facebook has some terms and conditions of how information can be used applied, and i think some additional options need to be noted, when filling out information on facebook, for "signin with facebook for your given website software" if user information may be used differently than what facebook terms and conditions are set to. 

if student data was treated in such a way. example.  myschool -> moodle, then student went to another school.. there moodle,  i could click sign on with previous school. and get information that way?  or perhaps send data from moodle to say facebook.  and then from facebook, back to another school moodle installation. 

with above... student owns the data, but institution has to deal with the data, collect it, use it, keep it for so long for records.  in that part it implies both student and institution owns the information.  but the institution... would have to hold to what ever "terms and conditions" that were applied at time of student accepting the terms and conditions.  the institution would not simply be able to ok.. we are changing our terms and conditions so we can sell information... without student coming back to institution and saying hey ok i agree to new updated terms.

issue is... yet again... most websites and software out there... have some sort of "catch clause" we are able to update terms and conditions without warning and without notice, at any time we see fit ((inserts better lawyer speak))....   this gives options to software developers to add and change information so they can grow.  but it also gives possible loop hole of institution suddenly changing terms and conditions so they could sell information. ((insert better lawyer speak examples)).

===============

in the... Siemens et al, 2011  conference video, it speaks about mutli data warehouses... moodle is already setup for multi data warehouses... from repositoriy plugins say google drive, different activities / resources / courses via moodle.net  to web conference software, to various chat software skype, irc, etc.., to various wide range of other plugins. 

but moodle does not have the "facebook" signin like information?  maybe some plugins? or it is simply implied through given yourwebsite -> moodle -> general terms and conditions?

===============

backup and restore and publishing to moodle.net  and keeping student data out of it... vs being able to give student data...  i would imagine would apply to this thread. 

while... www.yoursite.com/moodle *login as admin* -> site admin menu -> developer -> create test course,  can create test courses and test students.  it does not imply actual useful data of students going through a course.  

data analytic, implies ability to look at other students that have went through a course and/or activity,  and give suggestive other options to student via some computer AI errr education AI or academic AI.    while a k-12 may have say 500 students... they only have say 50 students in grade 8. 50 students is not a whole lot of data to base some things on.  and need for multi schools to combine there data sets.  to get a better algorithm / equation... for better learning experience through moodle.  example one school in 8th grade may only have one person that has this specific learning disability, but 10 other schools have zero,  but another 30 schools have an average of 1.2 with same learning disability.  if moodle software was able to say hey, this student has this learning disability.. lets say they are blind. lets give these optional activities / resources better suited to a blind person.   if person is deaf, then these activities / resources.  blind and deaf are easy... taking it further for more troubled students. and there different learning disabilities needing data sets to work from. **disabilities bad word not sure what else to use**

======================

going back to top of this post...  students GPL and information given out, vs stuff that needs to be withheld... and facebook signin via a given website or application... along with backup/restore of activities / resources / courses via moodle.net  and giving information away...

would a simple "yes/no" checkbox be good enough for different types of information...of letting students and/or parents and/or institutions decide what data to give away freely while still anomalously identify a given student?  

=======================

*situation grown out of control* 

medical information in the United States. *arghs*  scams upon scams, doctors selling information to drug companies and you would get a different call each hour from someone else trying to get you to do something buy something...  and eventually it had to come down to government placing a law on things.

what things can be put into place to keep situations from getting out of control?

Average of ratings: -
In reply to Michael de Raadt

Re: Paper review: Creepy Analytics and Learner Data Rights

by Scott Beattie -

Hi folks

Thanks for picking up on this paper - it's great to see these issues being discussed.

We have a draft Charter up on google docs,  I've enabled comments as well:

https://docs.google.com/document/d/1US9SRle7KDcO3o9uYMdRW7oZ6j_pe_O-Vsb17iIUc3s/edit?usp=sharing

Looking forward to reading more of your ideas,


Scott


Average of ratings: -
In reply to Scott Beattie

Re: Paper review: Creepy Analytics and Learner Data Rights

by ryan sanders -

Scott beattie.  some issues...

student being able to correct data. *shakes head no* some information is going to be within a database, and within log files. that are going to be pass limitations of being able to adjust.  

some information is going to need a lot more construct "database sql statements" and php programming for example. to be able to identify information. let alone being able to adjust the information. 

kinda asking a home builder, to dig out a rock under a concrete slab of a basement of a house. the excess work required. to do such a thing. i have some issues with. 

===============

<quote>2.5 open learning systems used when ever possible.</quote> that is pushing your wants onto things. 

<quote>3.1 Learning organisation have a licence to use learner data</quote> "license to kill" huh?   license bad word, though it maybe lawyer speak, that i am not understanding.

===============

there is no information for dealing with minors / legal guardians / adding on... old person taking course in something, and then for what ever reason someone else besides that old person has legal rights over information. 

minors are more likely bigger issue.  and possibly need for parent / legal guardian acceptance for some things, vs allowing minor to accept something.

===============

it does not cover... allowing a teacher. to send student to some other website. to participate in something.  were information may be captured about the student with or without teachers and/or students knowledge. 

being responsible, but things happen. 

Average of ratings: -
In reply to ryan sanders

Re: Paper review: Creepy Analytics and Learner Data Rights

by ryan sanders -

last min thought...

teachers freedom, to get it done. and be done and over with and move on. 

i could almost see a teacher get "scared" or use something like your document, as an excuse, of not expanding out side of some pre-defined information.  and in a way trap themselves.  due to information was not presented or done in x,y,z way of going about it. 

moral compass if you will / judgment call. 

===============

teaching for....

looking for information on other websites. that define "information is not sold/shared" on 3rd party websites and or looking for a specific,  say GPL like license number. (something to easy identify, without re-reading all the blah blah, and more blah blah blah lawyer speak), that many including myself may click right through and never care to look at. or less something comes up.

Average of ratings: -
In reply to Scott Beattie

Re: Paper review: Creepy Analytics and Learner Data Rights

by Michael de Raadt -

Thanks for sharing your charter document, Scott.

It's good to see you are going for an open and transparent policy. I'd be interested to know what you would need to put in place to achieve this.

Would you like more public feedback? We can certainly invite more.

Average of ratings: -
In reply to Michael de Raadt

Re: Paper review: Creepy Analytics and Learner Data Rights

by dawn alderson -

Hi,

I have drawn on stuff I thought about, immediately, when thinking in terms of ethics/data protection/rights for education/research and so on....maybe useful or not-because I know this is from a UK standpoint.  However, there maybe some pertinent points to build on.  I think the charter is a good idea.  One thing, I suppose which is tricky, as with all policy docs, is to attempt to avoid ambiguity in situating poss circs/vignettes as examples for application....but as I say this remains tricky across many policy docs as contexts for/in practice can vary.

Article 26 UNC HR

(2) Education shall be directed to the full development of the human personality and to the strengthening of respect for human rights and fundamental freedoms

The Data Protection Act of 1998 (UK) eight data protection principles concerning how personal data should be managed:

1.                         processed fairly and lawfully

2.                         obtained for specified and lawful purposes

3.                         adequate, relevant and not excessive

4.                          accurate and, where necessary, kept up-to-date

5.                         not kept for longer than necessary

6.                         processed in accordance with the subject's rights

7.                         kept secure

8.                         not transferred abroad without adequate protection

1.p

British Educational Research Guidelines (revised 2011)

https://www.bera.ac.uk/wp-content/uploads/2014/02/BERA-Ethical-Guidelines-2011.pdf

Voluntary Informed Consent (page 5)

11.  Researchers must take the steps necessary to ensure that all participants in the research understand the process in which they are to be engaged, including why their participation is necessary, how it will be used and how and to whom it will be reported. Social networking and other on-line activities, including their video-based environments, present challenges for consideration of consent issues and the participants must be clearly informed that their participation and interactions are being monitored and analysed for research.

Privacy (p7)

 25 The confidential and anonymous treatment of participants' data is considered the norm for the conduct of research. Researchers must recognize the participants' entitlement to privacy and must accord them their rights to confidentiality and anonymity, unless they or their guardians or responsible others, specifically and willingly waive that right. In such circumstances it is in the researchers' interests to have such a waiver in writing. Conversely, researchers must also recognize participants' rights to be identified with any publication of their original works or other inputs, if they so wish. In some contexts it will be the expectation of participants to be so identified.

26 Researchers must comply with the legal requirements in relation to the storage and use of personal data as set down by the Data Protection Act (1998) and any subsequent similar acts. In essence people are entitled to know how and why their personal data is being stored, to what uses it is being put and to whom it may be made available. Researchers must have participants' permission to disclose personal information to third parties and are required to ensure that such parties are permitted to have access to the information. They are also required independently to confirm the identity of such persons and must keep a record of any disclosures. Disclosure may be written, electronic, verbal or any visual means.

27 The Data Protection Act also confers the right to private citizens to have access to any personal data that is stored in relation to them. Researchers seeking to exploit legal exclusions to these rights must have a clear justification for so doing. 28 Researchers must ensure that data is kept securely and that the form of any publication, including publication on the Internet, does not directly or indirectly lead to a breach of agreed confidentiality and anonymity.


Average of ratings: Useful (1)
In reply to Michael de Raadt

Re: Paper review: Creepy Analytics and Learner Data Rights

by Michael de Raadt -

An interesting follow-up paper was published at last month's ASCILITE conference, sharing the experience of introducing an LA code of practice at a university.

Welsh, Simon and McKinney, Stewart (2015) Clearing the Fog: A Learning Analytics Code of Practice. In: Australasian Society for Computers in Learning and Tertiary Education (ascilite2015), 29 November - 2 December 2015, Perth, Australia.

Title Clearing the Fog: A Learning Analytics Code of Practice
Abstract Learning Analytics is an area of practice that impacts the legal and ethical obligations of educational institutions. New legislative regimes, growing concern about online privacy, and the affordances of the data being collected mean Learning Analytics could represent a risk to universities to the same extent that it represents an opportunity. These risks augur the need for institutions to develop formal practice and/or policy frameworks around Learning Analytics to define supported practice, actively manage risks and begin to build trust and ethical practice through transparency. There is a danger for Australian universities that the development of such “checks and balances” are not keeping pace with the technological advancements in this field. This paper outlines how one university is seeking to provide a frame for lawful and ethical practice of Learning Analytics through a Code of Practice.
Paper and details http://research.moodle.net/80/


Average of ratings: -