Performance of count_submissions_need_grading

Re: Performance of count_submissions_need_grading

by Alex Rowe -
Number of replies: 5

Below is some investigation from adding error_log((new Exception())->getTraceAsString()); to the count_submissions_need_grading function.

The same time we saw a log in the error log for the stack trace, there was a corresponding log in the access log for POST /lib/ajax/service.php?sesskey=xyz&info=core_calendar_get_calendar_monthly_view

The users which are triggering all the calls to the "needs grading" function are also students.

Below is the stack trace for one of the error log entries. For this one student, the AJAX request called the function 8 times before the entry was truncated.

[Fri Oct 23 12:51:41.438547 2020] [proxy_fcgi:error] [pid 20501:tid 140645424310016] [client IP HIDDEN:0] AH01071: Got error 'PHP message:
#0 /mod/assign/lib.php(1893): assign->count_submissions_need_grading()\n
#1 /lib/moodlelib.php(7719): mod_assign_core_calendar_provide_event_action(Object(calendar_event), Object(core_calendar\\action_factory), '103581')\n
#2 /calendar/classes/local/event/container.php(299): component_callback('mod_assign', 'core_calendar_p...', Array)\n
#3 /calendar/classes/local/event/factories/event_factory.php(42): core_calendar\\local\\event\\container::apply_component_provide_event_action(Object(core_calendar\\local\\event\\entities\\event))\n
#4 /calendar/classes/local/event/factories/event_abstract_factory.php(197): core_calendar\\local\\event\\factories\\event_factory->apply_component_action(Object(core_calendar\\local\\event\\entities\\event))\n
#5 /calendar/classes/local/event/data_...\nPHP message:
#0 /mod/assign/lib.php(1893): assign->count_submissions_need_grading()\n
#1 /lib/moodlelib.php(7719): mod_assign_core_calendar_provide_event_action(Object(calendar_event), Object(core_calendar\\action_factory), '103581')\n
#2 /calendar/classes/local/event/container.php(299): component_callback('mod_assign', 'core_calendar_p...', Array)\n
#3 /calendar/classes/local/event/factories/event_factory.php(42): core_calendar\\local\\event\\container::apply_component_provide_event_action(Object(core_calendar\\local\\event\\entities\\event))\n
#4 /calendar/classes/local/event/factories/event_abstract_factory.php(197): core_calendar\\local\\event\\factories\\event_factory->apply_component_action(Object(core_calendar\\local\\event\\entities\\event))\n
#5 /calendar/classes/local/event/data_...\nPHP message:
#0 /mod/assign/lib.php(1893): assign->count_submissions_need_grading()\n
#1 /lib/moodlelib.php(7719): mod_assign_core_calendar_provide_event_action(Object(calendar_event), Object(core_calendar\\action_factory), '103581')\n
#2 /calendar/classes/local/event/container.php(299): component_callback('mod_assign', 'core_calendar_p...', Array)\n
#3 /calendar/classes/local/event/factories/event_factory.php(42): core_calendar\\local\\event\\container::apply_component_provide_event_action(Object(core_calendar\\local\\event\\entities\\event))\n
#4 /calendar/classes/local/event/factories/event_abstract_factory.php(197): core_calendar\\local\\event\\factories\\event_factory->apply_component_action(Object(core_calendar\\local\\event\\entities\\event))\n
#5 /calendar/classes/local/event/data_...\nPHP message:
#0 /mod/assign/lib.php(1893): assign->count_submissions_need_grading()\n
#1 /lib/moodlelib.php(7719): mod_assign_core_calendar_provide_event_action(Object(calendar_event), Object(core_calendar\\action_factory), '103581')\n
#2 /calendar/classes/local/event/container.php(299): component_callback('mod_assign', 'core_calendar_p...', Array)\n
#3 /calendar/classes/local/event/factories/event_factory.php(42): core_calendar\\local\\event\\container::apply_component_provide_event_action(Object(core_calendar\\local\\event\\entities\\event))\n
#4 /calendar/classes/local/event/factories/event_abstract_factory.php(197): core_calendar\\local\\event\\factories\\event_factory->apply_component_action(Object(core_calendar\\local\\event\\entities\\event))\n
#5 /calendar/classes/local/event/data_...\nPHP message:
#0 /mod/assign/lib.php(1893): assign->count_submissions_need_grading()\n
#1 /lib/moodlelib.php(7719): mod_assign_core_calendar_provide_event_action(Object(calendar_event), Object(core_calendar\\action_factory), '103581')\n
#2 /calendar/classes/local/event/container.php(299): component_callback('mod_assign', 'core_calendar_p...', Array)\n
#3 /calendar/classes/local/event/factories/event_factory.php(42): core_calendar\\local\\event\\container::apply_component_provide_event_action(Object(core_calendar\\local\\event\\entities\\event))\n
#4 /calendar/classes/local/event/factories/event_abstract_factory.php(197): core_calendar\\local\\event\\factories\\event_factory->apply_component_action(Object(core_calendar\\local\\event\\entities\\event))\n
#5 /calendar/classes/local/event/data_...\nPHP message:
#0 /mod/assign/lib.php(1893): assign->count_submissions_need_grading()\n
#1 /lib/moodlelib.php(7719): mod_assign_core_calendar_provide_event_action(Object(calendar_event), Object(core_calendar\\action_factory), '103581')\n
#2 /calendar/classes/local/event/container.php(299): component_callback('mod_assign', 'core_calendar_p...', Array)\n
#3 /calendar/classes/local/event/factories/event_factory.php(42): core_calendar\\local\\event\\container::apply_component_provide_event_action(Object(core_calendar\\local\\event\\entities\\event))\n
#4 /calendar/classes/local/event/factories/event_abstract_factory.php(197): core_calendar\\local\\event\\factories\\event_factory->apply_component_action(Object(core_calendar\\local\\event\\entities\\event))\n
#5 /calendar/classes/local/event/data_...\nPHP message:
#0 /mod/assign/lib.php(1893): assign->count_submissions_need_grading()\n
#1 /lib/moodlelib.php(7719): mod_assign_core_calendar_provide_event_action(Object(calendar_event), Object(core_calendar\\action_factory), '103581')\n
#2 /calendar/classes/local/event/container.php(299): component_callback('mod_assign', 'core_calendar_p...', Array)\n
#3 /calendar/classes/local/event/factories/event_factory.php(42): core_calendar\\local\\event\\container::apply_component_provide_event_action(Object(core_calendar\\local\\event\\entities\\event))\n
#4 /calendar/classes/local/event/factories/event_abstract_factory.php(197): core_calendar\\local\\event\\factories\\event_factory->apply_component_action(Object(core_calendar\\local\\event\\entities\\event))\n
#5 /calendar/classes/local/event/data_...\nPHP message:
#0 /mod/assign/lib.php(1893): assign->count_submissions_need_grading()\n
#1 /lib/moodlelib.php(7719): mod_assign_core_calendar_provide_event_action(Object(calendar_event), Object(core_calendar\\action_factory), '103581')\n
#2 /calendar/classes/local/event/container.php(299): component_callback('mod_assign', 'core_calendar_p...', Array)\n
#3 /calendar/classes/local/event/factories/event_factory.php(42): core_calendar\\local\\event\\container::apply_component_provide_event_action(Object(core_calendar\\local\\event\\entities\\event))\n
#4 ...

We could have 800-1000 students logging in at around the same time to access resources/quizzes and the amount of times this function is called with the performance of the query isn't a good mix.

Average of ratings: -
In reply to Alex Rowe

Re: Performance of count_submissions_need_grading

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
It seems like the overview block is not essential to taking the quiz. Why not disable it during the quiz period?
Average of ratings: -
In reply to Tim Hunt

Re: Performance of count_submissions_need_grading

by Alex Rowe -
Hi Tim,

The overview block would be how everyone is accessing the their courses for the quiz. First logging in, going to their MY page, then to course page and the quiz.

I have been able to log the callgraph for a teacher in our staging environment and the "mod_assign_core_calendar_provide_event_action" function was called 86 times, and the "assign::count_submissions_need_grading" function 43 times.

Link to callgraph as it's too large to upload here

During exam periods, there are multiple exams per day or 1-2 weeks so we can't make any changes to the overview block but it seems unnecessary to call this function so many times, especially for users who are only students in courses and would have no assignments to grade.
Average of ratings: -
In reply to Tim Hunt

Re: Performance of count_submissions_need_grading

by Alex Rowe -

It looks like we might be able to edit the mod_assign_core_calendar_provide_event_action function in /mod/assign/lib.php as it may be calling count_submissions_need_grading when it isn't required, e.g. if they can't grade it anyway.

    LINE 1549: mod_assign_core_calendar_provide_event_action: /mod/assign/lib.php
    if ($event->eventtype == ASSIGN_EVENT_TYPE_GRADINGDUE) {
        $name = get_string('grade');
        $url = new \moodle_url('/mod/assign/view.php', [
            'id' => $cm->id,
            'action' => 'grader'
        ]);
        $itemcount = $assign->count_submissions_need_grading();
        $actionable = $assign->can_grade($userid) && (time() >= $assign->get_instance()->allowsubmissionsfromdate);
    } else {


The expensive query is being called in $itemcount then in $actionable it's checking if they can grade the assignment.

can_grade is only checking for the truthiness of has_capability('mod/assign:grade', $this->context, $user) so if the user can't grade then we don't need the count of submissions that need grading.

Would something like the below work better, by flipping $itemcount and $actionable then checking $actionable first.

        $actionable = $assign->can_grade($userid) && (time() >= $assign->get_instance()->allowsubmissionsfromdate);

        if ($actionable) {
            $itemcount = $assign->count_submissions_need_grading();
        } else {
            $itemcount = 0;
        }

count_submissions_need_grading already returns 0 if it's for a team submission so defaulting to 0 if the user can't grade would work similarly.

If using $actionable isn't the best for the if then storing can_grade in a variable first and using in both $actionable and the if is another option.

Average of ratings: Useful (1)
In reply to Alex Rowe

Re: Performance of count_submissions_need_grading

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
I don't know. This is not my area of the code. But, having reached this level of technical analysis (and it looks like you have found something worthwhile) you should probably move this to the tracker. https://docs.moodle.org/dev/Tracker_introduction
Average of ratings: -