Pierre,
Thank you again for all your comments.
I welcome warmly your willingness to participate, I has been sad about how little cooperation with other members of the community this project has had - it is partially my fault, too. What do you think you could help with, in addition to giving your ideas?
Your basic tenet is right. Doing usability as an one-man project is simply not enough. If Moodle aims to be taken seriously also in the future, usability needs to be built into the development cycle. But before that is the case, this is the best I could do during one summer.
There are many outstanding issues, which in part
Development:Quiz Usability portal/Where to go next with Quiz UI? Autumn 2008 tries to answer. In addition, there is the new tagging and searching functionality, which Tim is working on - the new design of the user interface for this functionality needs to be confirmed to be usable, too.
But more important than new features if we are to ensure quality, is testing, both usability and functionality wise, as well as other usability research: as a most basic instrument,
the user groups should be defined and documented. Moodle really needs to have the basic usability documentation and it needs to be maintained. This is the time.
HEAD vs CONTRIB
I need to do the conversion of code from 1.9 to 2.0 soon, in order to concentrate move fully on my thesis work. The problem then is, who is going to do the conversion from the code in CONTRIB, branched from the current Moodle 2.0 codebase, to the by-then-changed codebase.
For me, the question of whether the code should be included in Moodle HEAD at a given time is a question of evaluating risk. In short, the question can be expressed as: Will the new user interface serve the needs of Moodle's user groups as well or better as the old interface? This can be further evaluated by measuring the improvement or possible degradation of different properties in old versus the new version: usability, accessibility, code quality, compatibility with browsers.
Risks
Are we taking a risk when committing Moodle to the new UI by migrating it to HEAD of Moodle 2.0? If yes, can we justify the risk to be small enough compared to the benefits achieved?
There are, broadly, two kinds of risks:
- the functionality (architecture, algorithms, data structures, amount of bugs), and
- the design of the user experience (from abstract to concrete: personas -> scenarios -> use cases -> user interface).
The new interface is by no means complete yet. Still, I have reasons to believe that it is, as a single whole, serving most users' need better than the old UI. And your arguments against this seem few.
So far the ways to assure quality are as follows.
- To verify that the application has the correct functionality and workflow for the intended users, teachers, a small number of Finnish university-level teachers were interviewed about their working habits with exams, quizzes and the like. After that, feedback of the found scanarios was requested from the Moodle community. I should probably have made it easier for the community to take part since the discussion about the scenarios was not overwhelming - nevertheless, the process revealed some important aspects of teachers' work related to quizzes.
- As this project is mainly about the user experience and not the underlying art of programming, my aim was not to introduce any major new functionality. This was also to reduce the risk introduced by the project: as major data structures have not been changed, issues can be nothing but local in the UI. I have not changed any major algorithm implementations or data structures, and where I have made minor modifications to them, I have been in what I think is a close enough cooperation with Tim Hunt.
- The user interface was designed to comply better with heuristics, which are quite ubiquitous.
- Where greater changes to the interaction have been made, they have been confirmed against the usage scenarios, and some of them have been also usability tested. Is your argumentation based on the test results, available in the docs?
- I have also gathered feedback to all the changes during about half a year now (or if we count the first prototype, over a year) from the community.
I have discovered some things that still need to be fixed in the new UI in order to bring it to par with the old one, but they do not constitute a significant risk to me: improving things like keyboard navigation and tooltips, though important, do not require major changes to the logic of even the UI level, and can thus be added while the code is in HEAD.
"This is a too large project for one person."
What project in this world isn't? However, this is a fallacy (false argument): you can not judge how good something is by the number of people that have participated in it. Often, the contrary: it is a well-known fact of the software industry that it rarely helps a software project to just add more programmers.
However, I have cooperated closely with the Moodle community and one would think most the choices made have been confirmed by this point. I do not see any crucial compromise having been made. You do have the code available for testing already in the tracker. If testing is what you think more people are required for - putting code into CONTRIB does not seem helpful in this respect? Of course, once the code goes into head, fixes can be made by others, too.
"Has it been tested thoroughly enough? NO"
I warmly welcome you to find more people to do usability testing in Moodle.
But when it comes to functional testing, the functionality has changed very little in itself. For the most part, functionality that worked in Moodle 1.9 still works in 2.0 since it has not changed. I do not question that more testing needs to be carried out but I regard the code of sufficient quality to be tested in HEAD by as wide an audience as possible to really iron out
as many bugs as possible.
Rights management is one area where, though I meant to make no functional changes, it needs to be confirmed that everything is still in place.
The software has been tested on Firefox3, Opera9 and IE6 - if bugs are discovered on Safari or other versions of the aforementioned browsers, bugs on that level should be relatively easy to fix, if not otherwise then by browser-specific CSS.
Compatibility of javascript is a risk on IE7 and Safari, since those have not been tested.
"However the final product should conform to general moodle interface conventions and accessibility norm.
I think that the project should be quite polished either in the display as in its coding before CVS to HEAD.
If it was available as contrib so that we can CVS it, it will be easier to improve collectively. "
You are being very general about the "conventions". What general Moodle interface conditions does it not conform to? A lot has already been done to make it conform, and I still have not found any documentation of such conventions. What would you want to improve collectively if it were in CONTRIB?
Dividing the project in pieces
"I would suggest that the project be divided in clear objectives (sub projects) that can be put in 2.0 (or 2.1..) step by step but giving the time to test them."
This I could not disagree more with - the user experience, although affected by all the details and elements of the UI, is in essence one whole. This I have explained already in
Why can't we just make smaller fixes to the current ui? - in short, it does not make sense to do that, when the entire conceptual model is wrong and the UI is speaking a language foreign to the user. And on the other hand, even if we can redesign the UI to use a conceptual system more natural to novices, the advantage can be sabotaged if the elements of the UI violate user expectations too badly - whether this is the case can be evaluated for example by aligning the UI with usability heuristics.
In other words: I am not sure what the sub-projects would really be. The vast majority of the changes are not on the level of features, but instead only the UI has been changed, and the usability of the UI has already been tested - unless there is a plan to do more extensive *usability* testing, I see no reason to divide the project, since as the UI is a single whole it would risk breaking the usability of the entire UI to divide it into pieces.
"For example the order and paging innovation is only the numbering way to more easily move question in a large quiz.The questions being listed in a more compact form.
Could this be done by using the advanced parameters feature of moodleforms in the edit page ?etc."
The UI is not based on moodleforms (
PEAR QuickForm) since it is not quite flexible enough.
"For example the order and paging innovation is only the numbering way to more easily move question in a large quiz."
I do not understand this sentence. If you wish, you can express it in french so I can try to retranslate it, too? I hope you understand that this is not to criticize your English, but just to try to understand what is it that you want to say.
Accessibility
"Does it meet the accessibility and other guidelines? NO"
" The actual interface does NOT meet accessibility guidelines the question bank being the main problem( this cannot be used easily with a sreen reader)."
The question to ask in my opinion is whether the previous question bank did, either? If it did not, the new one is just improvement and although it is indeed crucial to make the changes involved to making the UI accessible, this is not related to the question of whether or not to move the new UI to HEAD: since no change has been made, no new risk is related to the new UI compared to the old UI. So my question is: where is the new UI _worse_ than the old one in terms of accessibility?
There are some relatively trivial TODOs in the
development docs related to keyboard usage already waiting to be implemented, but I invite you to inspect the accessibility more profoundly since though I understand the fundamentals, I am not an expert at accessibility.
Replies to further comments
"The use of tabs with icons, there incoherence (your edit the quiz but don't have access t the "real question bank or import and if you access them you don't see how to get back to the quiz editing "
Access to the real question bank has been included in the new UI since early prototypes, as a simple link. This has also been in part addressed at ..., but Tim argues that links to Question bank and back are enough. What do you think?
The icons have been removed for
IE in the newest version so the visual presentation of the tabs is no longer broken by IE. In other browsers it seems to work. However, in terms of usability it is useful to associate the icon with the term it is related to (edit, preview) so that users can easily comprehend the meaning of the icon when used elsewhere in the tab, too. This could be used in other parts of Moodle, too, in my opinion.
(I am also surprised that Moodle's login screen is fatally incompatible with IE 5, but this is unrelated)
"Understanding that for building a quiz you need first to build your questions and store them somewhere ( in convenient category) is a simple concept that is easy to understand even for students that can be allowed to create questions in 1.9."
And it is a concept supported by the new UI as well as the old one.
"However I have problems with the last proposal "Type question content directly in a text box inside the question box, right in the quiz editing screen"
at least for the calculated question type which is a 3 pages process but also for the minimum screen size necessary to do this. We should not forget that Moodle is used all over the world and not every moodle user can access a 22 inches screen..."
As you can see in the final product, inplace editing was not implemented at this time. The purpose was not to allow editing all the options of a question but just the question text, but there are several issues with even this, both technical and usability wise, so it was left out.
"P.S. Tim last year set a lot of constraint related to add a new question type. I think we should follow the same rules before modifying this interface."
Can you please provide a link so that I can check this out?
.
).
and this explain some of my comments .