Hi folks,
please note that there is an ongoing discussion of the Gradebook internal improvements that might have significant impacts on the Gradebook usage. We believe that the changes we are thinking about would bring useful features for advanced Moodle users while simple usage would remain simple.
There is a draft specification document at http://docs.moodle.org/en/Development:Gradebook_2.x_architecture that summarizes the planned changes. Now we would like to get a feedback from the community to see what you think of it. Thank you for constructive and thoughtful comments in advance.
In reply to David Mudrák
The myth of the default (Re: Request for comments: Gradebook 2.x architecture)
by Itamar Tzadok -
We are inclined to assume that defaults make it easier for the average user. This maybe true in many cases but not all.
IMO, the first problem of the gradebook even for a simple usage is that by default, evey activity with a grade is pushed into it. But in many cases many activities are for practice, often more than those which are for the course grade. This behavior overloads the gradebook with items that shouldn't be there. All solutions which involve states and degrees of visibility miss the point.
So, if this is not already in the developing spec, I suggest that the gradebook should start blank and the simple user will be able to add to it only the desired items (or select and add all items), just like adding questions to a quiz page (the principle, not the interface).
A possible approach for adding grade items is to use only manual grade items but make the manual grade items such that they may be set up to be based on a grade from an activity.
In reply to Itamar Tzadok
Re: The myth of the default (Re: Request for comments: Gradebook 2.x architecture)
by Nancy K Hoke -
I would love to see this addition - if the activity could have the option of adding to the gradebook yes or no - that would be great. I do hope that this one is added to the list of additions to the gradebook. Thank you so very much.
In reply to Nancy K Hoke
Re: The myth of the default (Re: Request for comments: Gradebook 2.x architecture)
by Itamar Tzadok -
Thank you Nancy. Your suggestion to add an option to the activity could work too. But consider this. In most cases, whatever you do in the activity settings with respect to the gradebook, you still have to go to the gradebook and make some adjustments. And in most activities the settings form is already overloaded and quite confusing, not to say intimidating, as it is. So perhaps instead of adding the option to the activity you could just go to the gradebook and select to add the activity grade item from a dropdown list. Instructors who do not use the gradebook (there should be quite a few) would have a simpler activity settings form to work with. Just a thought.
In reply to David Mudrák
Grades appearance (Re: Request for comments: Gradebook 2.x architecture)
by Itamar Tzadok -
This refers mainly to The grades appear in the gradebook immediately. I don't know if this will address all the issues with grades appearance but it seems to me that instead of ultra complex state machine to control grades inclusion or exclusion in aggregations, a simpler approach could be to allow the user to create more than one grades view. So, one view could be accumulative grade, another could reflect the actual course grading scheme, a third just raw grades etc. With the existing show/hide/until functionality, instructors would be able to control which views to make available to students and when.
In reply to Itamar Tzadok
Re: Grades appearance (Re: Request for comments: Gradebook 2.x architecture)
by Tim Hunt -
The gradebook supports multiple view - the different gradebook reports.
At the moment, all these different views display data from a single central data model. All the different activities feed data into that model, and the manually entered grades are stored there. The aggregations are computed within the model.
Therefore, having different views that display different aggregations would be difficult. The view would have to re-compute the aggregates with different settings. Not impossible though.
In reply to Tim Hunt
Re: Grades appearance (Re: Request for comments: Gradebook 2.x architecture)
by Itamar Tzadok -
From a user's perspective, in the course level there is only one view/report with two display styles, horisontal (the grader report) and vertical (the user report). This one view/report represents one way of aggregating the grades in the course.
To illustrate, this is like organizing the grades in one Excel worksheet and looking at this worksheet either as is or transposed. But with Excel I can add another worksheet with different columns and different aggregations of the same grades, and I can use this other worksheet either as read-only or read-write.
I haven't looked under the gradebook's hood so I take your word for it that the aggregations are static and not recaculated upon display (if that's indeed your word for it). Having different views that display different aggregations would not be difficult if the aggregations were recaculated upon display, right? Is there a substantial overhead?
In reply to Itamar Tzadok
Re: Grades appearance (Re: Request for comments: Gradebook 2.x architecture)
by Bob Puffer -
Allowing some room for my misunderstanding of what's being asserted, it is my opinion that aggregations for categories and course total are calculated on the fly. I have looked under the hood and see that everytime the grader report is refreshed, generate_grades() is called and when it hits a category or course item, aggregrate_grades() is called. The only exception to this is if Sum of Grades agg method is used.
Some courses feel the need for allotting a combined grade for all members of a group in a group activity. I don't know if this is to be incorporated in the assignment module or in the gradebook.
Kindly consider!
-Radhika
One one purely practical level, I have a request about the API for activities to pass grades to the gradebook. At the moment, the activity has to build an array userid => grade to pass to the update_grades function.
In the quiz, it would actually be easier for me (and potentially much higher performing) if I could call the API with the SQL for a query that returns a result set with two columns, userid, and grade.
See https://github.com/timhunt/Moodle-Question-Engine-2/blob/qe2_wip/mod/quiz/locallib.php#L614 (yuck, who wrote that long rambling method ). Focus, in particular, on the $finalgradesubquery bit in the middle.
Do we really need the complexity of the approval conditions? What about a single boolean flag in the grade item (is_released) that external events can toggle?
I suppose the danger is two competing external things trying to toggle it. E.g. the quiz trying to hide it because it is set not to display grades until after the close date, while the teacher keeps trying to manually open the eye in the gradebook.
I guess I'd think that in 95% plus conditions the approval is the grade itself. The approver is the one awarding the grade or setting up the module to award an appropriate grade. 100% agree this is unnecessary added complexity. This approval architecture doesn't address the single most-common scenario we encounter, "I want to see a running calculation of my grade to-date". This is pretty well met with the current gradebook and a couple tweaks to allow for extra credit in all agg methods.
I don't think it is a 'Problem' that teacher and students see different things in the gradebook. It is essential that teacher can quickly see raw grades as they come in before they are released to students.
And I think with the new data model, it is easy to achieve that, and letting Teachers also know what students will see. There is a bit of careful work needed to ensure that the state of everything is clear from the UI, but the back-end will have all that state information readily available to be displayed in whatever way we work out is the best way to display it.
Brief points:
- Storing all grades and settings in the gradebook, not the module, is good - providing there are good developer docs. I have never really understood what I should be doing in the quiz code. It has largely been monkey-see-monkey-do copying code from other modules
- I don't see any problem with storing grades <0 or >1.
Another technical API point:
Should we take the display of grades (as %, Float, scale, etc) out of the core gradebook code, and have a new, very simple sort of plugin grade/formatter.
That would be a subclass of a grade_formatter_base with a single format_grade method. It is interesting to think about what arguments that method would need:
format_grade($grade, $max, $isfinal, ...)
I am really unconvinced by the need for pluggable gradebook engines. It seems like unnecessary complexity. You only have one example of why it might be needed, and that could easily be handled by one configuration option that the core grade calculation code uses.
Hello,
I forwarded your proposal to some of our faculty that use the grade book heavily. Here is a response I got:
It looks as if they are working on some important improvements. The idea of having grades not show up until after they have all been entered is a good one, but part of a bigger issue that entering grades directly into Moodle (for a class of 60, for instance) is clunky and more time consuming that entering them in EXCEL and the importing them into the gradebook - but I don't see why this should necessarily be true.
I work with classes of 360 and I assure you with all the issues the gradebook may have, entering grades directly into Moodle isn't one. If your faculty are comfortable with Excel they should keep using it, and when the grades are ready to be published, they can import all the grades and feedback directly into the gradebook, by a few additional clicks. In many cases this method is by far more efficient than working directly on Moodle, even with small classes of 60, and regardless of what UI Moodle may introduce (I'd love to here why your faculty find this method clunky and time consuming). This also effectively resolves the issue that all grades should be made available online at the same time.
I do a lot of work directly on Moodle but I still do substantial work in the background in Excel and other apps, not only grades but also content (e.g. quiz questions). This not only makes my work more efficient but also my own backup file and means for transfering content from one year and its Moodle server to another without depending on adminatration services.
hth
I'm Sharon's faculty colleague who finds it unfortunate that it isn't easier to enter grades directly into Moodle. I use the same "few additional clicks" to upload grades from EXCEL - but by the time I've chosen the wrong format by mistake, and then go back to take out the extra line at the bottom of the EXCEL spreadsheet where I calculated an average grade but that was causing problems with uploading, and then upload again, this time successfully uploading the grades, and then repeat the whole upload process yet again in order to upload the comments associated with new grade items, and then I have to go in and tell Moodle what category the new grade items are in, and what the total points are, it has taken 10 minutes. And I'd rather spend those 10 minutes on something else.
And I could, if Moodle had a simple interface for entering grades for one particular item so that I didn't have to use EXCEL (by simple I mean that every grade requires three keystrokes: two digits and ENTER key, and no mouse movement, and above all, no autofill that fills in "100" if I type 1/ENTER).
I respectfully submit that the main problem with the gradebook as it stands is that it is NOT SIMPLE. I've read through the (sparse) architectural specification which introduces an even more complex back-end with the assumption that its complexities can be hidden from the user through a well-programmed UI. I've seen no evidence to-date that Moodle HQ-driven gradebooks have been able to successfully hide the complexities of the gradebook back-end. Having read all postings to date it seems a great many of them would agree that a lot of the suggested "improvements" are unnecessary complexity added in.
IMO, the best gradebook interfaces have come from community-driven initiatives. Yet... I see that word, "internal" in your first sentence which reminds me of the "delete everything upon unenrollment" fiasco. I'd encourage you to bring the whole discussion out into the community -- data model, etc. The people who understand best how the gradebook is used are out here and able to intelligently comment on the whole picture.
Sorry to be so contrary but all of this doesn't make a whole lot of sense from where I sit. The kind of issues our schools (used to) have are with:
- Aggregations: hardly anybody knows what it means and fewer how it works. Why are there so many agg types. We've limited it to three and are seriously considering limiting it to one -- Simple. Because...
- Weighting: is so easily misunderstood. Every grade has a weight -- either imputed by the teacher in the "weight" column (if using Weighted agg) or the grademax (if using anything else). If Sum of grades is fixed (which we've done) to allow inclusion of only graded items in its calculation and categories are allowed to accumulate maxes from children (which we've done) so they accurately represent the weight of the category before alteration, then "Simple" is the only agg method required, IMO. Allowing the default weight to be changed gets you "Weighted".
- Enormous confusion with nested course item/category/item layout of grader report. Flattening that out made a big difference in ease-of use. Categories and their children are color-coded in the headers. The ability to expand and contract categories (what a confusing interface) was absolutely not needed when we implemented a two-way scrollable interface so students and grade headers were always visible on the report. Also eliminated those pesky mouseovers. Would be nice to be able to drag or move items around on the grader report though from category to category
- Too much to edit in a big grader report is a real problem, fixed long ago by LSU with their "Quick grading" columns and rows.
- What the student sees. We made all the columns configurable for the user report and included some that weren't there before (weight, average, points (real, but cummulative), percentage, letter). Teachers easily turn off what they don't want their students to see.
All of this has been in use by our schools for four terms or more. I don't know what kind of uptake others have but we put up 850 courses a term and have no mandate that teachers need to use Moodle. 70% of the instructors use Moodle for at least one of their courses. 96% of the students use it for at least one courses. Of that total, 236 courses used the gradebook. I think that's a good percentage given the circumstances and I attribute it to students driving their teachers to know where they stand at all times and some slight fixes for a gradebook that's only kinda broken. Nicholas did an amazing job given the pressure he was under and the diverse audiences he was trying to satisfy. All-in-all the gradebook calculates accurately -- its just too complex for most people to use as it is.
I bet you haven't limited your teachers' Excel with all its aggregation functions, right? Excel is much more complex than the Moodle gradebook, and yet many who are lost in Moodle gradebook are comfy with Excel. Why? Because, to begin with, in Excel they can add only the few columns they need for the aggregation. But in Moodle they get everything else from the site and soon lose track of the few required columns. So they need to start using categories which they don't really need, and complex aggregation method so as to exclude these categories or individual items, and even formulas. And all they wanted is the sum of 4 grade items...
The solution is quite simple. Don't push anything into the gradebook. It should be like Excel. You go there and you add only what you need.
I think that teachers' adeptness with Excel is due to its familiar methods from their years of experience using it for grade management. The mechanics of spreadsheet management have remained pretty consistent for most of their careers. The Moodle grading interface has very different mechanics from Excel, so there is very little skill transfer. It is a new skill that must be acquired. I think that what a lot of teachers want, though it would be difficult to develop, is an embedded spreadsheet in Moodle that would behave like Google Apps, with familiar mechanics.
The Moodle grading interface is indeed different from Excel but not the very different and the grading mechanics is pretty much the same. I don't have any particular problem with either the interface of the mechanics but then again I'm probably highly skilled in these sorts of things. I only suggest that Moodle's pushing items into the gradebook is one thing too many and althoug this is done out of pure good will, it only backfires. It may perhaps be less pure but do more good, if Moodle will let the user pick which goes in and which stays out.
I can get along with the concept of adding only the grade items you want into the gradebook and leaving the rest off on the side somewhere.
Bob - you mention, "we implemented a two-way scrollable interface so students and grade headers were always visible on the report." -- Does this exist in Moodle 2.3?
Yes, our "grader report" freezes grade item headings and the student information column(s) so only the table of values scrolls -- both ways. It is called the LAE Grader (LAE standing for Liberal Arts Edition) and is available from CLAMP (Collaborative Liberal Arts Moodle Project), an organization of 30+ schools (almost all higher ed) in North America. The report is used by many (I hear from folks all over the world), does not change the calculations (though it provides for displaying accurate point totals if you desire) and is available for 1.9 and 2.x.
One more comment before I go to sleep. There has always been talk about a "centralized" grading interface so quizzes and assignments, etc, don't keep their own grades. That talk's gone on forever but nothing ever came of it so with very little ingenuity we altered mod/assignment/index.php to allow manual input of decimal grades and push them directly into the gradebook. If configured accordingly it will even ignore gradebook overrides (how much value did we get out of all that confusion?) so a teacher can update their grade from the gradebook or from the assignment interface and they don't really even need to know there's a difference (which some of them don't). That's 1.9 though. Its been in use for over a year and available in the tracker as a patch but gets ignored everytime another point release comes out. I'm not going to waste time finishing the 2.0 mods until we're ready to implement 2.0 unless someone from Moodle HQ gives some indication that the effort would be worthwhile beforehand.
This is not really related to the new architecture (I think) but just a comment:
- At present there are separate capabilities for grading within each type of module, e.g. mod/quiz:grade.
- It would be possible to change this to use a standard 'can you grade this' capability.
- Such a change would have advantages (shortens capability list, may simplify permissions) and disadvantages (cannot set up roles so that by default they can grade only certain types of thing).
I don't actually have an opinion either way - this just came up because I am specifying an enhancement that adds grading support to some OU custom modules. The current approach is to add new capabilities, which I'm doing; but Tim suggested I should post here to mention it.
--sam
I have read the draft specification and all the comments here. I somewhat appreciate the debate on the architecture but feel others are more qualified to lead that debate. From a practitioners point of view, the grade book 'appears' challenging to most moodle course practitioners facilitators. The coverage of the grade book a strength for power users, off putting for novices.
Second, our staff outlined they only discovered the grade book towards the end of process course design process, rather than incorporating grade book design as part of the course design process, (effectively using the grade book then became a secondary training concern for us)
My reflection is that grade books would be more readily used if somehow incorporated / highlighted into resource / activity creation. Eg Add to grade book YES/NO.
Second, there is an opportunity to raise the profile of the gradebook demonstrate to course facilitators by making a submission count or information a usable feature.
Terminology changes from 1.9 to 2 have been well received (eg page), is there a case within the grade book too?
After giving my reading further time to digest, there is one further point I would like to open to discussion.
The relationship between the course facilitator and the student and their assignment / activity submission.
Currently we use the assignment submitted block to notify work waiting to be graded. Can this aspect of the gradebook be brought within the core functioning (if it is not already?)
Hi David,
Is this post meant to discuss the list posted in the draft spects document, or are you open to considering and adding more features?
I do have a list as well
I would say don't forget those of us (many in the UK if not elsewhere) who do not use grades in the the way Moodle assumes. I do absolutely nothing in any of my (full time) teaching which awards a percentage grade or any other way of monitoring levels. I use only criteria-based assessment. The student has either done it or not. I am currently trying to use outcomes to monitor this sort of progress but currently the only way for a student to monitor their progress against those outcomes is the gradebook and that gets very cluttered and confusing.
TBH it doesn't matter whether the student passes an activity or not (especially learning activities) what matters is whether any outcomes covered by that activity are met. This gets more confusing when outcomes could be met in just one of a choice of activities. Perhaps rather than squeezing outcomes into the gradebook a dedicated outcome monitoring module is needed and then people with criteria-based assessment methods could just ignore the gradebook. If not then grading needs to become a lot more flexible and inclusive.
I would like to second Martin's point about criteria-based assessment. While I appreciate that an attempt to allow for criteria-based assessment has been made with Moodle Outcomes, it really isn't granular enough to provide the kind of information that would make criteria-based assessment manageable for most teachers (in the upper K-12 grades - I can't speak for the elementary level). To really track whether a student has met the required criteria (or Standards and Benchmarks in US lingo), it would be necessary for a teacher to be able to assign one or more criteria to every assessment question (not just every overall assessment), and then generate reports that would display student performance relative to each criteria. If that data could be captured, some really helpful information could be made available to teachers, students, parents, and administrators (e.g. - the student's mean performance relative to each criteria, the student's best performance relative to each criteria, the student's most recent performance relative to each criteria, whether each criteria has been adequately assessed). There is a huge push toward K-12 criteria-based assessment in the US, and the single biggest obstacle to implementing it is that it is extremely time consuming to do manually. I suspect that the only real means of making criteria-based assessment practical for teachers is via an LMS that builds the data from the assessments themselves (rather than manually evaluating all student assessments - formative and summative - and inputting them into a spreadsheet or stand alone grade book program). I have been considering trying my hand at writing a module that would accomplish this, but if those more talented than I would consider adding this functionality to the grade book (and all forms of Moodle assessment tools) that would be even better. I really think this is a major issue for K-12 schools, and I don't see any platform/program out there addressing it.