Machine learning model evaluation error messages

Machine learning model evaluation error messages

by László Bognár Dr. -
Number of replies: 8

Hi,

I am trying to evaluate a model but moodle 3.7 permanently says that there is an error in one of the course: "Invalid user". No more help. Nowhere.

I do not see any invalid user in any course (I use only one course in this trial). By the way why an invalid user would stop the evaluation process?

Do you have any idea what is next?

Attachment invalid user.PNG
Average of ratings: -
In reply to László Bognár Dr.

Re: Machine learning model evaluation error messages

by Elizabeth Dalton -
Have you deleted any users from the database? This can cause problems with linked records. Other than that, I’d need to know more about the model you were testing to help.
Average of ratings: -
In reply to Elizabeth Dalton

Tárgy: Re: Machine learning model evaluation error messages

by László Bognár Dr. -

Dear Elizabeth,

Thank you for interest. 

No, I have not deleted any users from the course which is valid for evaluating. 

I have just upgraded my earlier moodle version to 3.7 to be able to see the details of the new machine learning algorithms, mainly the the way of calculating the values of indicators from the moodle raw data and getting the data for other machine learning softwares. I am a professor of Applied Statistics at a university so I am not a real a moodle expert. (However I run my own moodle as an administrator for my courses.)

I set the end dates of all the other courses to remain only one course to be evaluated as a trial. This course was finished in July with 20 students. Many of them are not active students any more  but still valid. I set the minimum requirement in the gradebook and used the PHP model "Minimum grade to pass the course". I used 24 indicators mainly using learning activities rather than social ones. In the evaluation process the algoritm always stopped indicating the "invalid users".

My suspect is that this Analytics can handle only the course data originated in the new 3.7 version and not capable handling courses finished in a previous version. Is that possible?  The reason why I think is that I tried to start a brand new course in 3.7 and to produce new course data and in this case moodle reaction was "not valid sample data to analyze" and not the "invalid users".  May be the log data were not enough (I could produce only 3 fake students data in this trial.)

Dear Elizabeth, what do you think?

Looking forward to any comments.

László


Average of ratings: -
In reply to László Bognár Dr.

Re: Tárgy: Re: Machine learning model evaluation error messages

by David Monllaó -

Hi László,

This smells like a bug in the analytics API. As you say an error like this shouldn't stop the evaluation process, an invalid user should be skipped. The complete error log including the backtrace should be available in your error log. I've just tried to find the origin of the problem but there are too many possibilities. Could you please paste the error backtrace here?

Average of ratings: -
In reply to David Monllaó

Tárgy: Re: Tárgy: Re: Machine learning model evaluation error messages

by László Bognár Dr. -
Dear David,
Thank you very much for your help.

Attached I send the error trace. (I hope you asked for this).
Please tell me anything you need.
If you need I can give you an admin access to my site.
László


Debug info: SELECT id FROM {user} WHERE id IS NULL AND deleted = ?
[array (
0 => 0,
)]
Error code: invaliduser
×Stack trace:
line 1546 of /lib/dml/moodle_database.php: dml_missing_record_exception thrown
line 1522 of /lib/dml/moodle_database.php: call to moodle_database->get_record_select()
line 6386 of /lib/accesslib.php: call to moodle_database->get_record()
line 447 of /calendar/externallib.php: call to context_user::instance()
line 72 of /course/classes/analytics/indicator/activities_due.php: call to core_calendar_external::get_calendar_action_events_by_timesort()
line 166 of /analytics/classes/local/indicator/base.php: call to core_course\analytics\indicator\activities_due->calculate_sample()
line 475 of /analytics/classes/analysis.php: call to core_analytics\local\indicator\base->calculate()
line 408 of /analytics/classes/analysis.php: call to core_analytics\analysis->calculate_indicators()
line 326 of /analytics/classes/analysis.php: call to core_analytics\analysis->calculate()
line 198 of /analytics/classes/analysis.php: call to core_analytics\analysis->process_time_splitting()
line 104 of /analytics/classes/analysis.php: call to core_analytics\analysis->process_analysable()
line 270 of /analytics/classes/local/analyser/base.php: call to core_analytics\analysis->run()
line 576 of /analytics/classes/model.php: call to core_analytics\local\analyser\base->get_labelled_data()
line 192 of /admin/tool/analytics/model.php: call to core_analytics\model->evaluate()
Average of ratings: -
In reply to László Bognár Dr.

Re: Tárgy: Re: Tárgy: Re: Machine learning model evaluation error messages

by David Monllaó -

Thanks for the backtrace László.

There is a patch ready for the issue you identified and it is waiting to be integrated in the upcoming Moodle releases (3.7.2 and 3.8dev). You can track its progress in https://tracker.moodle.org/browse/MDL-66241. I would suggest you to watch the issue and upgrade your moodle site once the patch is integrated.

In the meanwhile, you can pull the patch yourself in a testing environment if you know how to use git.

Average of ratings: -
In reply to David Monllaó

Tárgy: Re: Tárgy: Re: Tárgy: Re: Machine learning model evaluation error messages

by László Bognár Dr. -
Dear David,
Thanks. I am happy and sad at the same time.
My last question: Will this bug also stop the evaluation process for the courses with new users or it causes problems only with users data originated in an earlier version?
B.L.
Average of ratings: -
In reply to László Bognár Dr.

Re: Tárgy: Re: Tárgy: Re: Tárgy: Re: Machine learning model evaluation error messages

by David Monllaó -
Hi László,

The bug stops the evaluation process when using the 'activity due' indicator in any students at risk model. Regardless of the course. You can remove that indicator and the evaluation process should work fine.
Average of ratings: -
In reply to David Monllaó

Re: Tárgy: Re: Tárgy: Re: Machine learning model evaluation error messages

by Nagy Gábor Zsolt -

Hi David,

Thanks for the patch. It has been applied on László's server and now it works fine.

Best,
Gábor

Average of ratings: Useful (1)