Data management - enhancement proposals

Data management - enhancement proposals

by sam marshall -
Number of replies: 4
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hello,

We would like to propose - and develop, unless anyone else is already planning it - some improvements to the way the app manages data. Specifically, a better user interface for data management.

We don't have time to develop this right now, it will be probably about February, but I thought I should post in advance, either in case people think it's a bad idea, or if core developers or anyone else are planning something similar and it might conflict, that kind of thing...

Open University context

Most of our courses are multiple gigabytes in size, because some of the content contains video. On the plus side, the fact that it is downloaded into the app (compared to e.g. hosting on YouTube) means it can be used offline, which is the key advantage of the Moodle app for our situation. On the minus side, this means students might run into storage limits.

However, I guess this applies to at least some other institutions.

Current situation

To quickly review the current situation as I understand it (apologies for any errors):

  • The Moodle app downloads content files from the websites if you press the Download button next to an activity or course.
  • It also downloads the files if you view the content individually without explicitly downloading it. For example, if you click into a Page activity that contains some images, these images will be downloaded just as if you had specifically clicked the 'download' button.
  • You can delete data for a single activity by choosing a delete option from the top right menu within the activity.
  • You can see how much data is used for an entire site, and delete that data, from the settings.
  • All of the data deletion only deletes files and not cached web service responses.

Problems

Although we have not yet made the app live at the OU, we have done a variety of workshops, user testing, interviews, etc. and identified problems with the above. Here are some of them:

  • When you download an activity, the download button disappears (later, if there is a change to that activity, it will show a 'refresh' icon to let you know you might have to re-download, but otherwise, it remains blank). This is a bit confusing to users who might not be certain if the activity downloaded or not.
  • You can download an activity from the course page, but you can't delete data from that page (you delete it from inside the activity). It might be hard to discover how to delete the data.
  • It is difficult to see how much data is used by an activity (you can do this by trying to delete the data, the confirm box tells you, but this means you can't even know it's worth deleting without trying to delete it).
  • It is impossible to see how much data is used by a whole section, or by the whole course.
  • It is impossible to delete data for a whole section of activities at once, or for a whole course (e.g. after you finished it).
  • When you download a large chunk of content, the app does not appear to warn you in advance if you have insufficient or low storage on your phone. (Note I am not certain about this, maybe it does!)
  • The app doesn't warn if you are downloading using mobile data. In the UK, some people use cheaper deals which only offer limited mobile data, e.g. 500MB/month, so it could be a problem if users accidentally download a large chunk of data.

Proposals

We propose the following changes to address these problems. I should say that these are approximate descriptions based on my current understanding and we intend to do user testing and so on, but this should be close enough to explain what I'm suggesting so that people can object to it if needed. smile These three are slightly linked but broadly independent.
1. Improve information on large downloads
When the user starts a large download, there may be an information box that gives them the opportunity to change their mind. It currently only tells you the size of the download. We propose altering this to ensure:

  • If you have insufficient storage on your device (and probably allowing say a 1MB margin), you get an error that informs you of this and doesn't let you start downloading the files. This is to avoid the situation where you actually use up all the storage, and other apps / phone functions break.
  • Otherwise, a warning box appears if you are downloading a large amount of data. The definition of 'large' should be different for wi-fi vs. mobile data, and depending on your phone's storage. For example, the rules might be something like: > 10MB on wifi, > 1 MB on mobile data, or > 20% of remaining phone storage. (Actual numbers TBC!)
  • The warning box should show the size being downloaded, the storage space available on your device, and a warning if you are not on wi-fi.
  • If wi-fi is disabled there might be a button to take you to your device's wi-fi settings (if this is easy to do; we've seen it in some other apps).
Examples (text also TBC):
You are about to download 7.4 MB over mobile data, because you are not connected to wi-fi. (Your device has 3.8 GB free.)
Or:
You are about to download 97.2 MB over wi-fi. (Your device has 624.0 MB free.)
2. Show icon for downloaded content
At present, there is a download icon (cloud with arrow), then when you click it and download finishes, it disappears.

We propose changing this so that:
  • A different Ionic icon (cloud with tick mark) appears after the item has been downloaded and is available offline.
  • If you press on this icon, a popup menu will appear which allows you to view the item (in case you just meant to click on the overall row), or delete the downloaded data. Deleting data does the same thing as if you choose that option within the activity.
Unchanged from now, if there is new data this would change to the 'refresh' icon, so you then can't delete the data (this way).
3.New course data management screen
A new screen could show information about all the downloaded data within a course.

  • It would list all activities where there is data downloaded, organised by section, showing the size of data for each item. Activities with no files stored would not be shown, and sections with no activities with files stored would also not be shown.
  • The total size for each section would be shown, and (probably at the top) for the whole course.
  • A button would let you delete all data for the course.
  • Delete icons next to each section would let you delete data for all activities in that section.
  • Delete icons next to each activity would let you delete data for that activity (exactly the same as deleting it from inside that activity).
Once this screen is implemented, it could be launched in two ways:
  • From the course menu (top right), Manage course data
  • From the menu you get when pressing on the downloaded icon (point 2 above).

Other things users want

The above three are practical things that I suspect we (developers at the OU) could probably code within the app, which is why I'm proposing them as relatively clear, defined items.

Users have also asked for some other things that I think may be harder or impractical or more of a fundamental change, or we just haven't prioritised quite as highly. (To put it another way, these are the ones where the product owner asked for it and I made a face.) Comments on these are also welcome.

  • For Android, option to store data on SD card instead of internal memory.
  • Show some kind of 'partially downloaded' icon (I don't know which... 'rainy'?) against an activity where the user has not explicitly downloaded it, but where they have visited some pages and it has stored some files.
  • Track whether data files were explicitly downloaded by the user or automatically retrieved in the process of viewing pages, playing videos, etc. If they were automatically retrieved, mark them as 'cached' only. Have a setting for the maximum cache size, and periodically clear out data that hasn't been accessed for a while.
  • (Nobody suggested this one but I was thinking about it) Deleting data, and the sizes etc, should really include the cached web service responses and not just the downloaded files, sometimes there could be at least a few hundred KB in there and it will mean the 'size accounting' is off...
The middle two of these sort of conflict i.e. if we get the cache behaviour working we can skip the half-cloud icon.

--sam

Average of ratings: -
In reply to sam marshall

Re: Data management - enhancement proposals

by Dani Palou -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Hi Sam,

overall I think you've got some great ideas and features in there!

I just wanted to clarify a few things:

  1. You say: It is difficult to see how much data is used by an activity (you can do this by trying to delete the data, the confirm box tells you). You should see the size in the delete option in the context menu, before clicking it. E.g. the 2.5MB in this image:

    context menu screenshot

    It can still only be seen when you're inside the activity, so the page to view all the usage in a course sounds nice!

  2. We already have some thresholds to display a warning to download/upload files, and they are different if you're on WiFi or 3G. For uploading files it's 10MB for WiFi and 1MB for 3G (see CoreFileUploaderProvider.LIMITED_SIZE_WARNING and CoreFileUploaderProvider.WIFI_SIZE_WARNING).  For downloading it's 100MB for WiFi and 10MB for 3G (see CoreConstants.WIFI_DOWNLOAD_THRESHOLD and CoreConstants.DOWNLOAD_THRESHOLD).  

  3. It's not easy to get the free space in the device. We used to display it in the app settings, but we removed it because:

    Desktop apps: Node's FS doesn't have any function for it, we'd need an external library to calculate it.

    Mobile: The Ionic Native File library has a getFreeDiskSpace function, but that function is NOT documented in the Cordova file plugin. There is a comment in the code saying:
    The getFreeDiskSpace plugin API is not documented, but some apps call it anyway via exec(). For compatibility it always returns free space in the primary external storage, and does NOT fallback to internal store if external storage is unavailable.
    We also saw that the result of the call wasn't reliable (at least in iOS).

Cheers,

Dani

In reply to Dani Palou

Re: Data management - enhancement proposals

by sam marshall -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Thanks Dani!

1. Not sure how we missed the size display in activities... Hmm... Maybe we were looking at an old version or the wrong activity type or something. (Our own plugin doesn't do this, we obviously need to copy the right bit of code there...)

2. I knew there were thresholds but didn't notice they were already different for wi-fi, those thresholds sound OK. I think the important change then is just to tweak the text to ensure users are warned they're on mobile data, basically people were concerned about users accidentally using up their quota. And the free space check would have been good but...

3. Oh dear, that's a bit of a problem. sad We just looked in the documentation and found the Ionic function and assumed it would work.

I searched and found two plugins that claim to do this but I have no idea how out-of-date or badly implemented they are...

http://phonegap-plugins.com/plugins/donvaughndcs/pgplugin-devicespace
https://www.npmjs.com/package/sqli-cordova-disk-space-plugin

Anyway, thank you for your comments, that's really helpful as usual. smile

--sam


In reply to sam marshall

Re: Data management - enhancement proposals

by Juan Leyva -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Hi Sam,

thanks for sharing this! 

As Dani mentioned, there are some technical limitations that prevented us to implement some of the features you mentioned in your post.

In any case, we are very open to supervise your work (if you finally find the time during February) so it could be integrated easily and quickly.

Please, ping us so we can coordinate both teams (OU and Moodle one)

Regards, Juan

In reply to Juan Leyva

Re: Data management - enhancement proposals

by sam marshall -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Thank you Juan, that's really helpful. We'll get in touch when we are in a position to do any work. smile

--sam