One of our customers recently had a spate of users who had uploaded coursework for assignments but then had forgotten to click the 'Save settings' button.
We could see from our web server logs that these users were clicking the 'upload' button, browsing around, uploading files, making directories, moving files around and renaming them, etc. But when they had finished, they were not then saving their changes and were instead navigating away to elsewhere in moodle, or just closing their browser window.
As a result, we've been looking at ways of mitigating this issue. One of the ways we've been looking at is to warn a user before they navigate away from a page if they've uploaded a file to it. As it turns out, the easiest way of implementing this for files is to implement it for forms in general. I've raised tracker issue MDL-31315 and put a patch together for this.
Unfortunately, whilst this works well in most browsers, a couple specifically do not support the relevant javascript calls to do this - the iOS mobile safari browser does not support this, and nor does Opera. From what I understand it's unlikely that they ever will. Additionally, Firefox doesn't let you customise the error message presented to users.
Do you think that this is a show-stopper, or is it better to help the majority of users from forgetting to submit content?
From a usability point-of-view, I'm aware that presenting the user with a modal dialogue which prevents them from leaving the page is generally frowned upon, but given the potential for missed coursework submission and other resulting consequences, I think that this is an acceptable solution.
I'd be grateful of any thoughts, comments, or (constructive) criticisms to this approach from a Usability perspective.
Thank you in advance,
Andrew Nicols
LUNS Limited
Usability: Helping to stop people forgetting to submit their work
by Andrew Lyons -
Number of replies: 16
In reply to Andrew Lyons
Re: Usability: Helping to stop people forgetting to submit their work
by Mark Johnson -
As discussed in this thread, I think the optimal solution is some sort of autosaving feature that saves a user's submission as they go.
In reply to Mark Johnson
Re: Usability: Helping to stop people forgetting to submit their work
by Andrew Lyons -
Hi Mark,
I really like the idea of autosaving (and was aware of it before), but I think that even with autosaving there is probably a need for a warning message. I assume that the autosave would take place periodically, and potentially also on the window close to ensure that users don't lose any changes.
However, as you've commented in the other thread, there are cases where you may not want the save functionality to act as 'Save and Publish' - you noted that several of your users tend to think of the 'Save and Display' as a non-publishing preview for example and hadn't realised that it made the content live. Additionally, if a user is making changes, but then decides that they don't actually want to make those changes, then they may just close their browser or move away (without clicking cancel) -- clearly it would be wrong for the autosave functionality to actually publish their changes when they never actually saved them.
As a result, users may have work auto-saved within their browser session, but not actually saved within moodle properly. In these cases, a warning message should probably still be displayed to warn the user that they haven't actually saved their changes and that they aren't visible to other users.
Similarly, if a user is uploading a file for an assignment, then the autosave will ensure that they don't lose their files, but it won't prompt them to actually submit them for assessment unless the autosave is an autosave and submit.
It would obviously be ideal if the autosave functionality knew when to save changes and publish them, and when to save changes and not publish them, but I doubt that we can interface with the users brain to know what they're thinking ;)
So all-in-all, I think that these methods should be able to complement one another where appropriate and shouldn't be seen as competing ideas.
Andrew
I really like the idea of autosaving (and was aware of it before), but I think that even with autosaving there is probably a need for a warning message. I assume that the autosave would take place periodically, and potentially also on the window close to ensure that users don't lose any changes.
However, as you've commented in the other thread, there are cases where you may not want the save functionality to act as 'Save and Publish' - you noted that several of your users tend to think of the 'Save and Display' as a non-publishing preview for example and hadn't realised that it made the content live. Additionally, if a user is making changes, but then decides that they don't actually want to make those changes, then they may just close their browser or move away (without clicking cancel) -- clearly it would be wrong for the autosave functionality to actually publish their changes when they never actually saved them.
As a result, users may have work auto-saved within their browser session, but not actually saved within moodle properly. In these cases, a warning message should probably still be displayed to warn the user that they haven't actually saved their changes and that they aren't visible to other users.
Similarly, if a user is uploading a file for an assignment, then the autosave will ensure that they don't lose their files, but it won't prompt them to actually submit them for assessment unless the autosave is an autosave and submit.
It would obviously be ideal if the autosave functionality knew when to save changes and publish them, and when to save changes and not publish them, but I doubt that we can interface with the users brain to know what they're thinking ;)
So all-in-all, I think that these methods should be able to complement one another where appropriate and shouldn't be seen as competing ideas.
Andrew
In reply to Andrew Lyons
Re: Usability: Helping to stop people forgetting to submit their work
by Mark Johnson -
Good points there Andrew. A thought occurs:
In those places I've implemented autosaving, an indicator (basically the standard "ajax loader" on a yellow background) appears in the corner of the page while something is being saved. Flipping that on its head, why not implement something similar that appears when a user first makes a change to make it clear that the change hasn't yet been saved. Perhaps something more obvious that my loader (red background? a sidebar? something next to the form elements?), but still passive. This could also work where an "autosaved draft" is implemented, the scenario you describe above where something's been saved in the browser but not in Moodle.
In reply to Mark Johnson
Re: Usability: Helping to stop people forgetting to submit their work
by Andrew Lyons -
Hi Mark,
This idea definitely has some merit and I like it. I agree that it would be good to show a passive marker when a form has been changed but not yet saved, but I'd still argue that there's a need for an active alert or warning of some kind to alert users when things haven't been changed.
I'd argue that this is maybe even more important in the case of autosaved drafts -- if a user has made a change which has been saved to draft and then navigates away without publishing; the next time they visit the page their changes will still present but they're still not published - this may not be entirely clear to the user.
I like the idea of the passive alert being next to the form elements, though I seem to recall you saying that form elements may have to be handled differently. If an element was not fully handled, it may present an odd interface if some items have the passive alert, and others don't. It may even be a bit jarring for users if some forms have passive markers beside each element, and some don't.
Thanks for the input,
Andrew
This idea definitely has some merit and I like it. I agree that it would be good to show a passive marker when a form has been changed but not yet saved, but I'd still argue that there's a need for an active alert or warning of some kind to alert users when things haven't been changed.
I'd argue that this is maybe even more important in the case of autosaved drafts -- if a user has made a change which has been saved to draft and then navigates away without publishing; the next time they visit the page their changes will still present but they're still not published - this may not be entirely clear to the user.
I like the idea of the passive alert being next to the form elements, though I seem to recall you saying that form elements may have to be handled differently. If an element was not fully handled, it may present an odd interface if some items have the passive alert, and others don't. It may even be a bit jarring for users if some forms have passive markers beside each element, and some don't.
Thanks for the input,
Andrew
In reply to Andrew Lyons
Re: Usability: Helping to stop people forgetting to submit their work
by Mark Johnson -
The different handling of form elements in this case would only apply to the way you determine whether they've been changed - I've not found the onChange event to be relaible for all elements in all browsers, some need an addition of onClick or onKeyPress. A simpler (but probably less efficient) way might be to simply store the value of a field onFocus, then capture it again onBlur and see if it's changed, although I dont know if that's any more reliable as I've never tried it!
In reply to Mark Johnson
Re: Usability: Helping to stop people forgetting to submit their work
by Andrew Lyons -
Hi Mark,
We've got this in testing in our production environment now so hopefully if this is an issue it should be flagged up. That said, so far so good. We've not seen any issues on any of the browsers we've tried (with the exception of Opera and mobile Safari).
I'll keep this in mind and try testing various elements a bit more in different browsers. Do you happen to remember which elements were being difficult?
Moving to use an onFocus and onBlur event also has other benefits - it should be possible to determine whether a change was made, then unmade. If we only set the tracking variable if it's empty, and we empty in the onBlur event if the current value matches the value in the variable, then we should be able to tell this.
My concern with doing it this way is that, if you have a form with many fields with a large volume of existing data, and a user changes many of them, you'll be storing a large amount of data in memory which could make the browser less responsive, or have a negative impact on other use.
Andrew
We've got this in testing in our production environment now so hopefully if this is an issue it should be flagged up. That said, so far so good. We've not seen any issues on any of the browsers we've tried (with the exception of Opera and mobile Safari).
I'll keep this in mind and try testing various elements a bit more in different browsers. Do you happen to remember which elements were being difficult?
Moving to use an onFocus and onBlur event also has other benefits - it should be possible to determine whether a change was made, then unmade. If we only set the tracking variable if it's empty, and we empty in the onBlur event if the current value matches the value in the variable, then we should be able to tell this.
My concern with doing it this way is that, if you have a form with many fields with a large volume of existing data, and a user changes many of them, you'll be storing a large amount of data in memory which could make the browser less responsive, or have a negative impact on other use.
Andrew
In reply to Andrew Lyons
Re: Usability: Helping to stop people forgetting to submit their work
by sam marshall -
Your approach sucks!
What, isn't that considered constructive these days?
The Firefox message is fine (I just checked it in Jira). For iOS and Opera, well, they get the same behaviour, right?
Basically I think this approach really does kind of suck (modal dialogs created by web pages are ugh) but there is no less-sucky alternative that I'm aware of. Really this is something that should be handled by the browser - there should be something like, window.setUnsavedData(true/false) - but unless that is in html5 and widely supported, not much option.
At least, that's true of the general case (e.g. data entered into text fields such as the forum post box). I'm not so sure regarding the file manager - I think there is a specific user interface issue there, because uploading a file seems kind of like you've already done an action so it seems like you maybe don't need to also save changes. It might be worth considering making it so that the file manager displays a highlight bar inside it (once you've added a file or deleted one or made any change) with text like:
Files changed. Save this form to confirm file changes.
or something.
--sam
In reply to sam marshall
Re: Usability: Helping to stop people forgetting to submit their work
by Andrew Lyons -
Hi Sam,
Thanks for the feedback. I do agree that modal dialogues suck but also that it's difficult to find a non-sucky solution in browsers. Browsers should really handle this themselves but I suspect that we'll have to find workarounds for quite a while yet.
Both iOS and Opera behave as they currently do - there isn't a way (that I've found) of popping up a warning and preventing the page change.
With regards to the file manager, there are some examples where a file manager is just another form element (e.g. creating a file resource). In these cases I think that it should have the same behaviour as other form elements which have been modified. Since the file upload screens for assignments are also moodle forms, we've have to find some way of displaying the message in an alternative manner in these situations.
The other solution I've been considering is to adjust the filemanager form element to automatically save any changes or file uploads, but again the filemanager is a form element like any other and is used elsewhere so this becomes difficult.
Thanks,
Andrew
Thanks for the feedback. I do agree that modal dialogues suck but also that it's difficult to find a non-sucky solution in browsers. Browsers should really handle this themselves but I suspect that we'll have to find workarounds for quite a while yet.
Both iOS and Opera behave as they currently do - there isn't a way (that I've found) of popping up a warning and preventing the page change.
With regards to the file manager, there are some examples where a file manager is just another form element (e.g. creating a file resource). In these cases I think that it should have the same behaviour as other form elements which have been modified. Since the file upload screens for assignments are also moodle forms, we've have to find some way of displaying the message in an alternative manner in these situations.
The other solution I've been considering is to adjust the filemanager form element to automatically save any changes or file uploads, but again the filemanager is a form element like any other and is used elsewhere so this becomes difficult.
Thanks,
Andrew
In reply to Andrew Lyons
Re: Usability: Helping to stop people forgetting to submit their work
by Steve Bailey -
Unless I'm mistaken, you can't upload from an iOS device anyway, unless you're referring to accessing repositories too.
In reply to Steve Bailey
Re: Usability: Helping to stop people forgetting to submit their work
by Andrew Lyons -
Hi Steve,
Correct, you cannot upload from an iOS device, but you can use the other repositories, and you can make changes to existing files (e.g. rename, create folders, move, and delete). You can also use other forms on an iOS device, and the changes I'm proposing would warn users when trying to leave those forms too.
Andrew
Correct, you cannot upload from an iOS device, but you can use the other repositories, and you can make changes to existing files (e.g. rename, create folders, move, and delete). You can also use other forms on an iOS device, and the changes I'm proposing would warn users when trying to leave those forms too.
Andrew
In reply to Andrew Lyons
Re: Usability: Helping to stop people forgetting to submit their work
by sam marshall -
(Automatically saving changes from a filemanager is definitely a no-go, at least globally. Would cause big problems. E.g. editing a wiki page, when you save it, it is supposed to create a new version - not edit files in the existing version.)
With regard to 'just like every other form element', conceptually yes it certainly is. I think the problem is that from a user's point of view it really isn't, because of the multi-stage process. Especially when you have forms which basically are just like a file manager and submit button and that's it. Arguably, any element which has a complex multi-stage popup process should have a similar 'edited, please save form' marker....
It's fine when creating a totally new resource, you aren't going to forget to save it because you know all the other stuff (name, etc) needs saving, but when editing one, could possibly have the same problems. I dunno.
I've only personally had the problem (as in, I myself forgot to save the form) when using a file-manager-and-submit-button type form. In a local plugin which, incidentally, I wrote. Still forgot to save it one time...
--sam
In reply to Andrew Lyons
Re: Usability: Helping to stop people forgetting to submit their work
by Andrew Lyons -
I've put together a quick screencast to demonstrate the form - http://goo.gl/Nl3y8
In reply to Andrew Lyons
Re: Usability: Helping to stop people forgetting to submit their work
by Kevin Wiliarty -
Our users have similar difficulties with the file manager. Without a warning, they may navigate away from the page believing their work is done. Only days or weeks later -- when the students can't find the readings or when the teacher can't find an assignment -- will they notice the problem. At that point it will appear to them as if we have 'lost' their files.
Any element of web design can be misused, but I believe that a helpful reminder to save work is a good use. Browsers that don't support the onbeforeunload event degrade to the default behavior. When the warning fails, no harm is done; when it succeeds (the majority of the cases) great good is done.
In reply to Kevin Wiliarty
Re: Usability: Helping to stop people forgetting to submit their work
by sam marshall -
I agree no harm is done when it fails. But to be more accurate about the success: in that case, great good and moderate annoyance are both done.
Certainly I find the behaviour in Jira very annoying - 90% of the time when I'm closing a tab I have typed something into, it's because I changed my mind and decided not to add that comment anyway, or to do it in a different way.
The other 10% of the time, though, it's a case when I might have lost data.
On balance I think it is preferable to have something that annoys me 90% of the time if it also prevents me from losing data 10% of the time, but it's not the case that there are only upsides to this behaviour.
--sam
In reply to Andrew Lyons
Re: Usability: Helping to stop people forgetting to submit their work
by David Price -
Andrew, in my opinion yours is a superb, pragmatic solution to an urgent problem. The idea about a coloured flag on the form is interesting but I think even that would be overlooked by most secondary school students. Any solution involving autosave seems months away and your efficient, tidy solution would stop the majority of careless clicking-away that gets blamed on Moodle, damaging its reputation. I'm intending to put your five changed php/js files into our production system to contribute to the testing, but the way you've designed the changes looks clean and efficient to me. In my opinion your work should be in core as it seems to becoming standard these days on many websites to expect a pop-up message when navigating away from an unsaved form.
Our students are used to using Moodle 1.9 and the need to remind classes to click on Save Changes dominates the use of Moodle 2 in the classroom. Students have commented that would expect a pop-up reminder if they navigated away before saving.
In reply to David Price
Re: Usability: Helping to stop people forgetting to submit their work
by sam marshall -
In case it wasn't clear from my other posts, I haven't looked at the code but based on the behaviour, I do agree this is an important improvement and should definitely go into Moodle 2.3. I voted for the issue.
--sam