Learning Analytics for Moodle

This project offers a privacy-friendly Learning Analytics solution for Moodle. The plugins integrate into Moodle courses and offer statistics to teachers and students.

This Learning Analytics solution consists of two plugins.

• local_learning_analytics: (this plugin) User Interface
• logstore_lanalytics: Logs the events to the database, you need to install this dependency first

Installation

Requirements:

• Moodle: Version 3.2 or higher (currently testing with 3.6 and 3.9)
• Database: MySQL/MariaDB or Postgres (other database types have not been tested)

Download

First, download the dependency (logstore_lanalytics). Afterwards, install this plugin.

Activating the logstore

After installation you need to enable the logstore plugin.

1. Go to Moodle Site administration page
2. In the Plugins tab, scroll down to Logging
3. Click on Manage log stores
4. There should be a row for the installed logstore plugin with the name Learning Analytics Log
5. Click on the eye icon (👁) to enable the log store.

The log store is now activated and will log events.

After installation and activation of the logstore, there should be a link in each course menu called "Coure Statistics" leading to the following page (COURSE_ID is the course id of the corresponding course):

https://MOODLE_INSTALLATION/local/learning_analytics/index.php/reports/coursedashboard?course=COURSE_ID

Import data from logstore_standard

After installation, all statistics are empty as no data has been logged so far. But your Moodle site might log data through Moodle's own logging system, the logstore_standard_log. The logstore plugin offers a simple way to import that data by using the import.php script. It can be called from the shell like this:

$ cd MOODLE_PATH/admin/tool/log/store/lanalytics
$ php cli/import.php

This will immediately start the import process. Instead you can also call php cli/import.php --help to see a list of possible options. You can for example only import the last X weeks or import only events starting from a specific ID to limit the amount of data you import.

Configuration

You should configure both plugins. The logstore plugin has options related to logging data and writing data to the database. The local plugin has options related to displaying the data.

Plugin local_learning_analytics configuration

The settings page can be found in:

Site Administration -> Plugins tab -> Local plugins category -> Learning Analytics

Options

The plugin has the following options:

• status: One of show_if_enabled, show_courseids, show_always, hide_link, disable. This value sets whether the user interface should be activated and whether a links should be shown in the navigation. By default, the link and page are visible if logging is enabled for the course. You can use this option, if you want to enabled logging in all courses, but only want to enable the user interface on specific courses.
  • Option show_if_enabled: Show navigation link and page if logging is enabled for the course
  • Option show_courseids: Show navigation link and page if course is specified below via course_ids
  • Option show_always: Show navigation link and page for all courses, even if logging is disabled for the course (only enable this, if you already logged data before)
  • Option hide_link: Hide navigation link but keep the page enabled for all courses (only if you know the link, you can access the page)
  • Option disable: Hide navigation link and disable the page for all courses. This will completly disable the User Interface for everyone.
  • Option course_customfield: Add customfield to all course settings. When selected teachers can select in their course settings if the link should be shown. You should read the notes below before switching to this option.
• course_ids: To be used with the status option show_courseids to only show the UI in specific courses. Example: 10,153,102.
• navigation_position_beforekey: Allows to specify where in the course navigation the link to the page is added. By default, the link is added before the first section node. Example value: grades to be shown before the link to grades. You can find the key of a node in the navigation by using the developer tools. Right-click on a link in the navigation, press Inspect and check the attribute data-key of the corresponding a element.
• dataprivacy_threshold: This value determines how many "data points" a "data set" needs to contain before the data is displayed. See the data privacy section below for more information. By default, the value is 10.
• student_rolenames: In case the role(s) for students/users in a course is not student, you can specify the corresponding role name. In case there are multiple roles that students have, use a single comma. Example: student,customrole. By default, the value is student.
• student_enrols_groupby: Option to allow merging of multiple courses with the same shortname or fullname in the parallel/previously heard courses. By default, the value is course->id which will not merge any courses by comparing their name.
• setting_dashboard_boxes: Determines which boxes are displayed in the dashboard, in which order and how big the boxes are. The specification is in the format reportname:width, separated by commas. A line has a maximum width of 12, after which it breaks. Example: learners:8,activities:4 displays two boxes in the dashboard, where the first one is much wider than the second one. The value only needs to be changed if other subplugins are installed or if the layout of the dashboard needs to be changed.

Option course_customfield

This feature requires Moodle 3.7. In older version of Moodle, this option will be unavailable. As described above, the option course_customfield adds a Moodle customfield to the settings page of your course. Technical details how this works can be found in the Moodle Wiki on Custom fields API.

Before switching to this option, you should be aware of the following:

• When switching from another option to course_customfield, the courses specified in course_ids will automatically be set to activated.
• When switching from course_customfield to another option, the (previously created) customfield will be deleted and with it the information which course had this setting enabled. That means, before switching from this setting back to another setting, you want to take a look at the database table customfield_data and see which course had this setting enabled (course = instanceid column, activiated = intvalue column)

Unfortunately, Moodle does not allow to use multi-language strings here, so we cannot provide separate German oder English texts for this option. You can however, edit the customfield on your own. You find the options under Site administration -> Courses -> Course custom fields. You can change the description of the field as well as the name of the category. But you should not delete the category or customfield and should not add additional custom fields to the category.

Option dataprivacy_threshold

Our plugin logs no personal data. For privacy reasons, you probably still want to restrict when aggregated data is shown. You can do this by setting the dataprivacy_threshold option. This option will hide datasets with less than dataprivacy_threshold data points. Depending on what is shown, the dataset will be completly hidden or shown as < 10 (with the threshold being 10). Whether data is hidden or not depends on the following:

• Analytics related to data that is unknown to the teacher (like what other courses the users are enroled into) is not displayed if the number of data points is below the threshold.
• Analytics related to data that is known to the teacher (like his own learning material) is displayed as < THRESHOLD

You should ask your data privacy officer what value should be used here. From our experience, 5 or 10 are common values.

Examples

Let's consider an example for each case. Assuming, the value is set to 10, this will have the following effects:

• Unknown data: In the Learners report that shows the number of courses users have heard before, only courses with at least (>=) 10 users in common will be shown. Courses with less than (<) 10 users will not be shown at all.
• Known data: In the activities report that shows the number of clicks for each learning materials, clicks are shown as < 10 if the number of clicks is less than 10.

Aggregated Data

In addition, aggregated data (like the number of total clicks on all quizzes) is rounded (down) to a multiple of the threshold. By doing that, the teacher cannot use the aggregated data points to esimated the data points of a specific activity.

Data storage

What is being stored?

When an event is triggered inside of Moodle, the following data is logged by the logstore plugin:

Table: logstore_lanalytics_log

In addition, there are helper tables, that do not store data-privacy related data and only exist to speed up queries or to minimize storage requirements.

Table: logstore_lanalytics_evtname

This table serves as a reference for the type of actions. It maps the eventid from the above table to the actual eventname. The table contains only two columns and is simply used to minimize the needed storage space inside the database.

Resulting storage size

Our Learning Analytics plugins were developed to log as little data as possible and to be as space-efficient as possible. A single table row needs 38 byte. Additionally, you should expect that you need roughly twice as much space after incorporating storage space for database indexes. All other (helper) tables are not worth mentioning and will need less than a few MB of space.

Some real numbers: The plugin has been used for multiple semesters at the RWTH Aachen University. The RWTH has roughly 45,000 students. In one year (two consecutive semesters), about 115 million rows were inserted into the log table. The needed storage was around 6.9 GB (including data and indices).

Data Privacy

This plugin was developed with data privacy in mind. It does not log any user ids. All data is logged anonymously. As this plugin logs no personal data, you don't need the consent of users to log the data.

After consultation of our data privacy officer, the following information was added to our data privacy statement (Datenschutzerklärung):

Im Rahmen von Learning Analytics werden anonymisierte Statistiken zum Zugriff auf Lernräume gespeichert. Bei jedem Aufruf in Moodle werden dabei folgende Daten ohne einen Bezug zu Nutzenden geloggt:

• Typ der Aktion (z.B. ob ein Quiz durchgeführt wurde oder ein PDF runtergeladen wurde)
• Uhrzeit (sekundengenau)
• ID des Lernraums, in dem die Aktion durchgeführt wurde
• Betroffener Kontext (z.B. die ID des Quiz, das gestartet wurde)
• Genutztes Betriebssystem (z.B. Windows oder Linux) und genutzter Browser (z.B. Firefox oder Edge), detaillierte Versionen werden nicht gespeichert

Sämtliche Daten werden anonym gespeichert und lassen keinen Rückschluss auf Nutzende zu. Die Statistiken können in den teilnehmenden Lernräumen über einen Link in der Navigation von allen Teilnehmenden der Veranstaltung abgerufen werden. Aggregierte Daten werden daher in den Statistiken erst dann angezeigt, wenn mindestens 10 Datensätze vorhanden sind. Andernfalls wird nur "< 10" angegeben.

Access / Capabilities

There is a single capability local/learning_analytics:view_statistics that decides who is allowed to view the course statistics. By default, the following roles have the cabability (as defined in access.php):

• student
• teacher
• editingteacher
• manager

That means that, by default students and teachers both have access to the statistics. This was done on purpose for maximal transparency.

Citing in Papers

The Learning Analytics for Moodle plugins were developed by Thomas Dondorf at RWTH Aachen University in the context of his PhD dissertation:

Learning Analytics for Moodle: Facilitating the Adoption of Data Privacy Friendly Learning Analytics in Higher Education

If you use this project as part of your research or want to reference it, please use the following BibTex:

@phdthesis{Dondorf:844544,
    author       = {Dondorf, Thomas}, 
    othercontributors = {Nacken, Heribert and Persike, Malte},
    title        = {{L}earning Analytics for {Moodle}: Facilitating the Adoption of Data Privacy Friendly Learning Analytics in Higher Education},
    school       = {Rheinisch-Westfälische Technische Hochschule Aachen},
    type         = {Dissertation},
    address      = {Aachen},
    publisher    = {RWTH Aachen University},
    reportid     = {RWTH-2022-04002},
    year         = {2022},
    doi          = {10.18154/RWTH-2022-04002},
    url          = {https://publications.rwth-aachen.de/record/844544},
}

Sample:

Dondorf, T. (2022). Learning Analytics for Moodle: Facilitating the Adoption of Data Privacy Friendly Learning Analytics in Higher Education (Dissertation). Rheinisch-Westfälische Technische Hochschule Aachen, Aachen. https://doi.org/10.18154/RWTH-2022-04002

Provided by

The plugins are provided by: