(Archived forum) Moodle research

Hi,

I think that all participants in moodlemootUS15 already know how to start adding resources or activities to a Moodle course. but I wonder how long does I take to a novice to find the way to start adding something to a course without help.

MoodleCloud recently presented by Martin, is a great idea but how easy is for a new user to perform without support the most important task for a teacher when taking the control of a virtual course?

My apologies if my post is not related to what you were looking for.

Best regards from Spain. I'd like to go to moodlemootUS15 this year but its a a bit expensive from where I live. Enjoy your moot and share as much as possible through twitter.

Toni Soto

Take a non Moodle user who is comfortable using web sites and ask them to create a quiz with one instance of each Core question type in it. See which question types they click on the help icons/ do google searches/ask for help with.

Time how long it takes to create each question.

Count how many "errors" they make (you decide what an error is).

A day later get them to do the same exercise without access to any help.

All the question types they cannot figure out get them in a new "advanced" category (or perhaps a disabled and invisible category) of question type......

Edit their profile and add a photo of themselves.

Create activities. 

Add resources.

Set up a gradebook and enter grades. 

Look at course participants and send a message out to one or all of them.

Clearing inline style tags out of atto to try make your content usable or professional looking - always fun time

Organising content in a large course with drag n drop.

Also lots of fun on touch screen devices.

Trying to delete the course news forum they didn't request or add.

Sending a message to a student, group of students or the whole course.

Figuring out why some items in the breadcrumb are clickable - normally the page you are already on - and other things in the breadcrumb are not clickable.

• Set up their profile
  
• Choice activity
• Locate the Moodle video channel on You Tube (so they can get access to further help)
• add in the online user block
• check what sort of enrollment is set
  

Each of these would help them to understand our community ideas of

• learners expect to know who is teaching them and something of their background
  
• learners are encouraged to be interactive not passive
  
• teaching them to be even a little bit self-sufficient is always a good idea (our guys find moodle docs straight off a little daunting)
• teaching them how to tell who can enrol in their course.
  

Thanks for asking.

From a user support perspective, here are the things I get asked about a lot:

• Gradebook configurations:
  • Setting up a points-based vs. weighted gradebook with categories
  • Hiding/displaying/editing what students see in relation to the gradebook (e.g., running totals)
    
• User activity:
  • Locating details such as last date of participation, activities accessed, etc. for a single user.
    
  • Viewing a single user's forum posts together for scoring.
    

Change the number of sections in a course

Show and hide resources

message users who have / have not completed a certain task

embed a video from youtube

embed a video from elsewhere (TED, Vimeo, etc)

view students' logs

set up conditional access activities

set up any sort of form (feedback, questionnaire, etc)

• Time to construct a gradebook that reflects the proper weighting and organization of a course.
  
• Using a sample course with data to learn how to grade/ provide feedback on assignments, forums, and other activities.
  
• Building a branched lesson.
  
• Create an instructional design model for delivering content that is practical to duplicate.
  
• How long it take for them to see how awesome Moodle is.
  

Tasks I would ask to check for a student are:

1. Add or update a profile picture.
   
2. Navigate to a different course when they are in an activity
3. Determine which activities are due
4. Submit an assignment
5. Message another student or teacher

Thanks for all the excellent suggestions, guys.

I've tried to collect your ideas together and group them. I've put them in an implied order.  Within our time limits, we would probably only be able to achieve one from each list and probably only down to organising the course.

Would you agree that this represents a priority/order for basic teacher tasks?

---

User profile

• Edit their profile and add a photo of themselves.
• Set up their profile
• Add or update a profile picture.

Viewing participants and messaging them

• viewing the course participants and messaging them
• Look at course participants and send a message out to one or all of them.
• Sending a message to a student, group of students or the whole course.
• message users who have / have not completed a certain task
• Message another student or teacher

Adding a resource

• uploading files by drag and drop
• embed a video from youtube
• embed a video from elsewhere (TED, Vimeo, etc)
• Locate the Moodle video channel on You Tube (so they can get access to further help)
• Add resources.

Creating an activity

• create a quiz with one instance of each Core question type in it
• setting an assignment (ie a task for their students to submit)
• Choice activity
• set up any sort of form (feedback, questionnaire, etc)
• Using a sample course with data to learn how to grade/ provide feedback on assignments, forums, and other activities.
• Building a branched lesson.
• Create activities.

Organising the course

• Organising content in a large course with drag n drop.
• Change the number of sections in a course
• Show and hide resources

Working with the Gradebook

• Set up a gradebook and enter grades.
• grading on Moodle
• Setting up a points-based vs. weighted gradebook with categories
• Hiding/displaying/editing what students see in relation to the gradebook (e.g., running totals)
• Time to construct a gradebook that reflects the proper weighting and organization of a course.

Viewing student information

• Locating details such as last date of participation, activities accessed, etc. for a single user.
• Viewing a single user's forum posts together for scoring.
• view students' logs
• add in the online user block

Tasks we're not likely to be able to achieve in the testing environment

• set up conditional access activities
• check what sort of enrollment is set
• Determine which activities are due
• Submit an assignment
• Navigate to a different course when they are in an activity
• using  Moodle with their mobile or iPad/tablet
• Create an instructional design model for delivering content that is practical to duplicate.

Thanks to everyone who has been involved in the preparation for this usability study so far.

The study will take place at the University of Minnesota usability lab on Monday 3 August between 8am and 3:30pm (see this in your timezone) with some reflection afterwards.

Would you like to be an observer, viewing user actions through live video and eye tracking and helping to comment on a collaboratively created document during the usability study? You can can be part of this online and you don't have to be available for the entire day. If so, please email me on michaeld@moodle.com so I can arrange for you to be added to the live session.

For those interested in the plans at this stage, see the attached documents including the scenarios developed from your suggestions.

For those unable to make it in person or participate online, we will hopefully be able to share videos and files here afterwards.

I'm really pleased to see hq engaging in real user testing.

https://moodle.org/mod/forum/discuss.php?d=200924

The tracker issues seems to have been closed at the time, so good to see you guys are now doing this and look forward to it being reopened.

https://tracker.moodle.org/browse/MDL-32544

Once you get an idea of the terrible usability of moodle and  benefits of user testing it would perhaps be beneficial to reach out and see how other open source projects overcame this?

https://moodle.org/mod/forum/discuss.php?d=176557

This is a report about the usability study that was held at the University of Minnesota on Aug 3, 2015.

Thanks to the people who contributed to the design of this study by providing tasks.

The setup in the usability lab at UofM was really impressive and the staff who ran the study were true professionals. We would not have been able to achieve the insights gained without them. The lab was divided into two rooms, one for a participant and one for the observers. There was a one way mirror separating the two rooms so observers could see into the room.

The study involved six participants. The participants were screened and were expected to have little to no experience using Moodle. Participants were each asked complete a set of five scenarios (linked here earlier). Briefly, the scenarios covered:

• editing the user's profile and sending messages to students and a fellow staff member,
• creating a page resource and embedding a video,
• creating a quiz including a number of question types,
  
• creating an assignment and
  
• creating a choice activity.

Textual data for things like profile fields, questions, descriptions, options, etc, were provided to the participant in a text file that they were directed to use. Other files, such as a profile image were also provided.

There was still a range in participants' abilities and success. In the 20min allowed, some participants managed about two scenarios, others more and one participant was able to complete all six.

Participants used a computer that was equipped with eye tracking to show observers where they were looking. The screen itself, including the user's action, and the view of the participant was also shared with observers. Perhaps the most insightful information was the verbal feedback from participants who were asked to "think aloud". All of these streams were recorded.

The scenarios were deliberately vague, allowing for multiple possible solutions and avoiding leading participants. When participants got stuck (and they all did), they were able to access a "helpdesk", which was a phone line into the observers' room. (Imagine being able to ask Martin Dougiamas for support whenever you got stuck with Moodle.)

Observers noted issues during the sessions. At the end of each session, issues were discussed and collected centrally. At the end of the day, a review of issues was conducted. Some issues were consolidated while others had recommendations recorded against them. I have attached the issues identified to this post as a PDF.

The issues will be reported in the Moodle Tracker so they can be worked on. You will be able to see them together in the epic MDL-51211 and you can follow the progress of these issue from there.

Would you like to be involved further?

• UX Researchers are welcome to view the complete recording to see if they can find more issues. Before you can view the recording, you will need to agree to a code of conduct. A link to the recording can be requested by email to me at michaeld@moodle.com.
  
• Developers are welcome to be involved in resolving the issues in MDL-51211.
• Users are welcome to watch issues within MDL-51211 and test out the functional changes to see if they make Moodle more usable.