Moodle in English: Moodle "Health Center" feature

Hi everyone,

I recently had this idea of a Moodle "health center", accessible only by the admin, which in essence would be a script that:

Detects common (or maybe uncommong) problems that we know about and cause confusion to the admins and/or users of a site.
Notifies the admin of said problems.
Provides instruction on how to solve them.

A classic example is the "whitespace present after ?> in config.php breaks redirects etc" problem. Another issue could be the php memory_limit. Or maybe the use of blank or weak passwords for MySQL.

For Phase One of this project, I would like to gather a few ideas of such problems (problems that admins could easily solve by themselves if only they knew what the cause is) to incorporate into this script so that I can post a working example. Can you guys brainstorm and fling a few candidates at me please?

After this, we pass into the realm of "could that be???" features... obviously these would take a little time to get implemented, but please read along and imagine the end result.

In Phase Two, we can substitute the PHP error_handler() with one of our own. This handler would, in addition to normal behavior, log PHP's notices and warnings somewhere (db?), and would give the admin a summary like:

There were 522 notices and 78 warnings generated by Moodle code.
Click here to view the warnings (a simple list stating file, line, message for start; later we can have some more complex manipulation for our convenience).
Click here to reset the counters (i.e., "I fixed those, let's see what else there is").

This would obviously let people give the developers lots and lots of feedback, because the error_handler can save any information we tell it to save at the time the error is generated, and people can just copy paste it to us without needing to understand or reproduce the problem (have you seen what kind of info a debug_backtrace() gives?

). I believe this would be a big step towards a rock-solid Moodle. But it gets even better...

In Phase Three, it's possible that we will have worked out a system for Moodle to "communicate" with the servers at Moodle.org, for example "click here to install Klingon language pack", or maybe "click here to install Excellent New Module", or "click here to update your copy of Moodle" (OK, this is excessive tongueout

). If I 'm not mistaken there is already some preliminary support for doing this for the language packs.

Well, what prevents us then from extending this communication and making it bilateral? We can have a "click here to submit debugging information to moodle.org" button, or even an option for automating this through cron.php. Result? Once this system penetrates to end users, if there is a PHP warning somewhere we get to know about it immediately! Plus, by keeping simple statistics like "X out of Y Moodle installations reported this problem, click here to break down by Moodle version"... do I need to continue? wink

In Phase Four, we will have integrated this system with the update procedure so that it can actually update itself! (my preliminary design is built in such a way that those "problems" that the health center detects do not need to be hard-coded).

What this means is that as soon as we find a serious bug or something like that, Martin can update moodle.org's servers with the info. Then, administrators around the world can have Moodle itself alert them to the fact that there is an issue they should address ASAP, and provide instructions to do that (or even say "click here to download and install this patch").

update.moodle.org anyone? big grin

So, to wrap this up: I 'd like to get some feedback from people (both end users and of course MD and developers) regarding this idea. I 'd also like anyone who can think of something appropriate for Phase One to suggest it, so that I can gather a few issues and present an initial version of this feature.

Jon

Average of ratings: -

Re: Moodle "Health Center" feature

by Mark Stevens - Saturday, 5 February 2005, 11:25 AM

Jon,

You are simply brilliant

This is a great idea. Moodle smart software that can fix itself, or at least tell you what you need to do to... Moodle software update! I think the option should be left open to the user to "Click here to submit error report to Moodle." Again, you're brilliant

Average of ratings: -

Re: Moodle "Health Center" feature

by John Papaioannou - Saturday, 5 February 2005, 11:37 AM

Obviously these "technical details" like who chooses to send information, when, and what kind of information are just that, details. It's the concept I wanted to present.

Thank you for your kind words. blush

Average of ratings: -

Re: Moodle "Health Center" feature

by Jan Dierckx - Sunday, 6 February 2005, 2:15 PM

Good idea.
Something I've noticed recently is students requesting an account with strange usernames. I've changed the 'make a new account' text to display something like: only use alphanumerics in your username, but some of them really seem to like: ::..._-_johnny&marina_-_....:: as their preferred loginname.
Of course their email client displays the confirmation link incorrect, so they can't get a Moodle login. This is something teachers are not aware of, so it really belongs with the admin stuff.
On second thoughts: I could change the login code to make sure only alphanumerics are accepted.
Nevertheless: a screen showing students who have requested an account but not activated it, could be usable. I work in a small school so I could ask them / their teachers personally about what went wrong.../ why they haven't activated the login.

Another thought: I once tried incorporating a link check script.
Could this be part of the Moodle health center?
A script which the admin runs to check all resources for broken links? Of course there should be some way of displaying them to the teachers to fix them. sometimes these scripts take a lot of time to run, that's why I think it would be better if the admin runs such a script.

Average of ratings: -

Re: Moodle "Health Center" feature

by Martin Dougiamas - Sunday, 6 February 2005, 4:36 PM

About the accountnames, look for the setting: "extendedusernamechars"

admin >> config >> variables

By default it's off ...

Average of ratings: -

Moodle should have ... wait, it's already there!

by Jan Dierckx - Sunday, 6 February 2005, 4:59 PM

... and I got angry because of students not reading the instructions about alphanumerics !

Average of ratings: -

Re: Moodle "Health Center" feature

by koen roggemans - Sunday, 6 February 2005, 5:14 PM

For inspiration on what common errors the healthcenter should search for, there is lots of info in the forums.

I think the most common are:

the slasharguments thing
wrong rights on the datafolder, database or the filterfolder
"invalid argument supplied for_each"
a damaged table in the database
email not being sent out for some reason.

As far as I can see, most of the errors are during installation or just after installation. Verry often caused by not reading or understanding the installation instructions and faq well enough, wich is very normal, or having too few knowledge of the system on wich Moodle is installed.
It would surely be a great help, such a healt system: there are a lot of helpfiles and faq's for Moodle and reading/understanding them all before starting is not a natural way to go for a curious human brain knipoog

.
So a system wich points out what to read/do when things go wrong will be a big emprovement. The installationscript is already a very big step in this direction.
Splendid idea.

Average of ratings: -

Re: Moodle "Health Center" feature

by John Papaioannou - Monday, 7 February 2005, 8:53 AM

Thanks a lot Koen! approve

Average of ratings: -

Re: Moodle "Health Center" feature

by W Page - Monday, 7 February 2005, 3:32 AM

Nice idea Jon!

WP1

Average of ratings: -

Re: Moodle "Health Center" feature

by Martín Langhoff - Monday, 7 February 2005, 6:08 AM

Jon,

I think it's a great idea. It's clear where you are coming from and the problems you want to address affect most installations at one point or another.

Some things are rather hard to detect (corrupt indexes under mysql, for instance), but if we can find a way, it'll be great.

We already have some warnings that show up in admin/index.php if something is wrong. If cron.php hasn't been run there's a nice warning, for instance.

Do you think that's a good place in the sense of UI? Are we served better by /admin/index.php or by a separate page?

Phases 3 and 4 scare me a bit. If a moodle 'admin' user can escalate his privileges to 'shell user' access we are in trouble. Do you see phase 3&4 having the ability to install/modify php files? That's a genie I'm not sure we can let out of the bottle

I say lets do it. The hard part is detecting those situations, so let's do that first -- write snippets of code that detect anomalies. Some can be interactive, and some will need to be in cron.php because they take time to run.

Where to report them... well, we can put them here or there and I don't really mind. For email reporting, the virus scanning already has a similar question (admin "id=1" or all admins).

cheers, ML

Average of ratings: -

Re: Moodle "Health Center" feature

by John Papaioannou - Monday, 7 February 2005, 8:21 AM

Hi Martin,

Indeed it won't be easy to detect all problems, of course, some of them could well be impossible to detect from PHP, but I think that it's going to be good enough if we can detect as many of them as we can. Plus, in phase 2 of the plan, we will be detecting problems we don't even know about -- that's going to help Moodle a lot even if it doesn't help the users who encounter those problems "on the spot".

The warning about cron in admin/index.php is one of the things I 've put into the health center detection; it's definitely very very good to have that somewhere, but the problem is that it's not a comprehensive solution. You cannot print 200 notices in admin/index.php, even if there is good reason to. Also, there are some things that are becoming apparent as I 'm delving into this that would require enough UI elements to decide not to clutter index.php with. And finally, I think it's better as an idea to be able to say "go to /admin/security" (or something like that) "and Moodle will give you a full report of possible issues".

As for the genie of phases 3 and 4, it's true that we cannot do that without careful planning and implementation. Obviously I 'm not saying "we should do that as soon as we find a way to"; rather it's "we should do that if we can find an acceptably safe way to".

I 've just about finished a preliminary version of the script, and I will be committing it to CVS HEAD shortly so that people can take a look. It's fully self-contained, so it should be easy to tweak or remove from CVS at any point.

Average of ratings: -

Re: Moodle "Health Center" feature

by Martín Langhoff - Monday, 7 February 2005, 9:26 AM

Great! Are you thinking of getting it ready for 1.5?

Average of ratings: -

Re: Moodle "Health Center" feature

by John Papaioannou - Monday, 7 February 2005, 10:50 AM

Good question!

There are a couple of other things, affecting existing features, that I feel I should really take care of before 1.5 comes out. And unfortunately, there are also things competing with Moodle for my time. So you should probably define "it" clearly before this can be answered, because this early in the project ideas come in faster than they get out. Where should we draw the line? You know how it is... wink

Average of ratings: -

Re: Moodle "Health Center" feature

by John Papaioannou - Monday, 7 February 2005, 8:51 AM

OK, the first version is in!

Points of interest:

It's not linked to from anywhere; you need to go to /admin/health.php manually.
All the strings are not in the lang files, for obvious reasons. If this stays in HEAD, then we can migrate them normally.
The CSS styles are inlined in the file, again so that people can edit them without bothering the main installation. Those will also be moved to where they should if it's globally approved.
Right now, when you visit the page it runs all the checks it can. Because this might take lots of time in the future, we might show only a "saved" set of results and have an "update to reflect current status" button or something. Lots of room for UI and administration interface improvement, let's see what people think.

Ears wide open for feedback! big grin

Jon

Average of ratings: -

Re: Moodle "Health Center" feature

by Tony Hursh - Monday, 7 February 2005, 9:28 AM

This sounds great, Jon. I'm looking forward to checking it out.

While this doesn't cause an immediate error per se, maybe it'd be good to have the health center display some kind of warning if the system was using the root user for MySQL, particularly with a blank password.

I notice from the config.php files that get posted on here that many people are doing this. It's scary.

Of course, MySQL warns you about this when you install it, but it might be good to present a friendly reminder.

Average of ratings: -

Re: Moodle "Health Center" feature

by koen roggemans - Monday, 7 February 2005, 9:40 AM

Great: the script found that my cron isn't working and php is not configured properly to let moodle manage the memory limit (wich is normal, becouse I don't pay attention to it on my testserver as long as it works

)

Good job Jon cool

Average of ratings: -

Re: Moodle "Health Center" feature

by Martin Dougiamas - Monday, 7 February 2005, 9:50 AM

Note that admin/index.php should have already been reporting both of these ..

I like the script, Jon, but how about encapsulating it all in a function of some kind so we can run it during install.php (which does a lot of the same stuff already and in most cases is the only time we need to check many of those things) and also from admin/index.php so the admin can find out about server changes as soon as possible.

BTW, my config.php doesn't have spaces...

Average of ratings: -

Re: Moodle "Health Center" feature

by John Papaioannou - Monday, 7 February 2005, 10:55 AM

I think it shouldn't be very hard to encapsulate this (code from health.php):

$problem = new $classname;
if($problem->exists()) {
    $severity = $problem->severity();
    $issues[$severity][$classname] = array(
        'severity'    => $severity,
        'description' => $problem->description(),
        'title'       => $problem->title()
    );
    ++$problems;
}

wink

I 'd appreciate some suggestions on how exactly to break down the code into one-responsibility-only functions that can be used from elsewhere. There's lots of evil things to think about, and the champions are valiant but few! big grin

Average of ratings: -

Re: Moodle "Health Center" feature

by Martín Langhoff - Monday, 7 February 2005, 12:49 PM

Take a look at the logic testing for cron not having run (in admin/index.php), or imagine testing for config.php being writable by apache using is_writable().

Most of the problems that we see hitting users time and again are testable with a few lines. With that in mind, could we simplify the approach? Wrapping a one-line test in class clothing seems a bit heavy.

Average of ratings: -

Re: Moodle "Health Center" feature

by John Papaioannou - Tuesday, 8 February 2005, 6:11 AM

I think wrapping the tests in classes is necessary.

It offers encapsulation of the tests
It helps not define something like 1000 global functions (in the future)
There's really no performance decrease

IMHO it's a no-brainer...

Average of ratings: -

Re: Moodle "Health Center" feature

by Mike Churchward - Monday, 7 February 2005, 10:27 PM

I like this idea, especially if we get to the point where we can choose which tests to run (or groups of tests?).

Also, it might be useful to come up with a plug-in structure to make plug-in type tests. That way, add-on modules might be able to add extra tests that could help with their maintenance / debug.

mike

Average of ratings: -

Re: Moodle "Health Center" feature

by W Page - Tuesday, 8 February 2005, 12:34 AM

The ability to test "Plug-ins" would be very useful.

WP1

Average of ratings: -

Re: Moodle "Health Center" feature

by Petr Skoda - Tuesday, 8 February 2005, 2:30 AM

My +1 for the plug-in concept.

I would like to add the ability to compare the installed moodle files with official distribution checksum file - it would reveal changed, missing, added or non-readable files. I have seen so many problems here concerning mixed versions or non-readable files.

skodak

Average of ratings: -

Re: Moodle "Health Center" feature

by Tony Hursh - Tuesday, 8 February 2005, 3:00 AM

That's an excellent idea. Besides helping to solve incompatibility problems, it could be used to generate a list of any files that have been customized so you know which ones need to be modified again after an upgrade (I try to keep good records of this, but invariably some minor tweak that I made gets forgotten and I don't find out about it until someone complains). Optionally displaying the diffs would be nice, too.

Average of ratings: -

Re: Moodle "Health Center" feature

by Martín Langhoff - Tuesday, 8 February 2005, 5:07 AM

CVS?

Average of ratings: -

Re: Moodle "Health Center" feature

by Tony Hursh - Tuesday, 8 February 2005, 10:05 PM

Yeah, that would work.

For that matter, you could save an untouched copy of the files and run plain ol' diff.

I was envisioning something a little less turbo-geek, though.

Also, those who run Moodle on Windows don't have CVS or diff installed by default.

Average of ratings: -

Re: Moodle "Health Center" feature

by Vicktorya Stone - Monday, 1 September 2008, 8:42 AM

now, from 3 and a half years later, from someone also not wanting more turbo geek (i.e., I have yet to get my cron confirmed out) -- so, I run health center. Which is a (great idea). However, what do I do with the information ...

(see below),

1st, the 2nd error, Session errors. (Session errors could be due to cookies not set in the IE browser which I opened to run the line ...)

However, what do I do now about Slash Arguments.

Health Center

Health Problems Detected!
Uploaded files: slasharguments disabled or not working
Slasharguments are needed for relative linking in uploaded resources:

slasharguments are disabled in Moodle configuration

can not run automatic test, you can verify it here manually

Session errors detected
Session problems were detected. Total count: 15

Uploaded files: slasharguments disabled or not working Slasharguments are needed for relative linking in uploaded resources:

slasharguments are disabled in Moodle configuration

can not run automatic test, you can verify it here manually

Solution Slasharguments are disabled, automatic testing not possible.

IIS:

try to add cgi.fix_pathinfo=1 to php.ini

do NOT enable AllowPathInfoForScriptMappings !!!

slasharguments may not work when using ISAPI and PHP 4.3.10 and older

Apache 1:

try to add cgi.fix_pathinfo=1 to php.ini

Apache 2:

you must add AcceptPathInfo on to php.ini or .htaccess

try to add cgi.fix_pathinfo=1 to php.ini

_________________________

My host is DreamHost, and they run Linux ... so what does that have to do with Apache or IIS?

I also doubt any of this may be up to date anymore. I am having a helluva time finding good information (let alone timely). I've read installation (but Dreamhost has one-click so that is easy; and I've read the Dreamhost wiki, which makes me uncertain about a few things). And I've read hundreds of posts here, each usually ending in a dead end, or so many multiple technial options that don't apply, I get weary and try again the next day. It's been months now.

I can't even tell anymore where to look or post, in the forums or the wiki, etc. I find posts from 2005 -- what happened to the posts about "Health Center". I don't mean to rant, I am just L O S T.

And, I have a site up, which I JUST need to get the cron working, and something baout the PHP ini -- I guess I do that is, based upon hints in various posts.

When I go to Installation FAQ (which seems a likely place to visit for this stuff), like "do this: 1, 2, 3, " I see nothing on 'what you must do', but a list of 'frequently posted problems', with fragmented solutions. I probably shouldn't post this, but I am at nearly wits end in FINDING anything useful to help me set this thing up clearly. (And, you know what, I've been working on computers since 1982. So, not exactly a newbie, although TRYING to maintain my 'end user' status.

Average of ratings: -

Re: Moodle "Health Center" feature

by Dan Marsden - Monday, 1 September 2008, 9:37 AM

the line you need to look at is "slasharguments are disabled in Moodle configuration "

normally this is done in IIS installs - you wouldn't do this on an apache box.

you can re-enable it in the admin settings here:
Site Administration > Server > HTTP

Average of ratings: -

Re: Moodle "Health Center" feature

by Marc Grober - Wednesday, 3 September 2008, 6:17 AM

but you are ranting..........
and there are people trying to help you.....

the results you saw in Health Center indicate possible reasons for artifacts the probe saw, and identified possible sources of the problem. moodle docs provide info on setting slashargs (see http://docs.moodle.org/en/Installation_FAQ#Uploaded_files_give_.22File_not_found.22) which is what in effect Dan suggested.

You will likely get more help if you can provide additional details and if you respond to the offers of help you receive.

Having done dozens of installs with DH I am familiar with most of the gotchas, but without details of exactly what you have done and what exactly the issue is it is difficult to help.....

Average of ratings: -

Re: Moodle "Health Center" feature

by Vicktorya Stone - Wednesday, 3 September 2008, 7:55 AM

Hi -
thanks, Dan, and Marc. I do appreciate your responses. (Sorry I was, at best, impolite if not boring ... I know.) I'll try to get my act together.

Meanwhile, I think the cron is running, on the wget line --
and on this slashargument, I've now checked marked it, which should disable it, as default is yes.

If I have further questions (when!), I'll do my best to document the needs and actions better, and try more research first.

appreciation,
Vicktorya

Average of ratings: -

Re: Moodle "Health Center" feature

by Martín Langhoff - Tuesday, 8 February 2005, 3:34 AM

Code cannot audit itself (checksum-wise) reliably.

That's why you need separate code, on different media. People who run tripwire do so from WORM media like a CD. Or they'll use a packaged system like Debian, and run the debian file checksums from a "Live CD" against the known-good MD5 fingerprints.

I really really really like this proposal -- I think we need to do this. Now, there's an easy part to it (creating a new page, with a new super-flexible framework) and a hard part (writing many tiny snippets of code that detect stuff but aren't triggered with false alarms).

If we just do the hard part and skip the easy part, what's the downside?

Hard parts:
- run reliable permission checks on all of moodle, cross platform
- offering a useful suggestion when module mismatches are detected (already being detected)
- detecting trailing spaces in config.php reliably. One ofthe problems with this is that if they are there and breaking sessions, admin never gets to login and see admin/index.php!
- database index corruption -- detecting it or guessing it
- detect and warn about "changeme" user - problem: if the user deletes it from the UI the user isn't really deleted.

What do you guys think?

Average of ratings: -

Re: Moodle "Health Center" feature

by Martin Dougiamas - Monday, 7 February 2005, 9:49 AM

Thanks, Jon, big vision!

Apart from the considerable security/privacy/performance problems of the later phases, I do like the possibility of an optional trackback-style system.

However, we had another plan in place already (bug 1342) which I think might be better ... that was to modify all the error() and notice() functions to call strings from lang/xx/error.php ... the error function would also include a link to moodle.org using the same errorcode/stringname where we could keep a database/discussion about that particular problem. We could certainly make this link a form that submits a dump, too (good idea).

The idea of this is that the whole Moodle would be self-diagnosing and contextual ... all we need to do is place the checks at appropriate points and admins would send us the data when they needed help.

Many of these "health" checks are currently done in install.php and admin/index.php, so that admins get warnings when they need them - we can certainly extend these.

As for automatic updates, yes, downloading of language packs is planned next (and the 1.5 code supports these). Module code could come sometime after that, but it does require quite radical changes to how moodle code is organised (ie it will all need to be stored in dataroot, called by "stub" scripts in the main Moodle directory). Not something I want to jump at given the long list of basic issues with things like interface and security that need to be solved now ...

Average of ratings: -

Re: Moodle "Health Center" feature

by John Papaioannou - Monday, 7 February 2005, 10:43 AM

This is ultra cool... I 'm very happy to have generated this response so far!

Martin, was just thinking about some of these points myself!

First of all: there are two kinds of errors we can come accross in Moodle. The first kind (let's say Type A) is where we get to call error() or notify(), i.e. the code has detected some "strange" things happening and it doesn't know how to deal with them. Type B errors, on the other hand, are PHP E_WARNING and E_NOTICE messages which mean that even though the code has tried to think of everything, it has nevertheless failed. My proposal doesn't replace what you describe (how to handle Type A errors), but it extends it.

For Type A errors, I agree that we should migrate from error() to some new mechanism which features "error codes", localized error messages and trackback to moodle.org. The discussion for bug 1342 certainly features many brilliant ideas! approve

Think about how this applies to Type B errors: we can't have localized error messages, since the error is unknown to us. However, we can have error codes (file name and line will suffice, or maybe we can concatenate them and md5() for extra privacy, even if it has other drawbacks). And since we can have error codes we can also have trackback to moodle.org. (This obviously means that after a release is made, some errors are found. We then set a trackback URL/discussion in moodle.org for those people who have the problem but cannot upgrade yet).

Both of these types of errors can be logged and displayed at the Health Center. This has at least four advantages that I can think of:

it's a comprehensive solution,
it lets the site admin know what's going on (remember that it could very well be that other users are in fact encountering these errors),
it's a permanent record of what happened, so admins are not forced to keep the info in their head or elsewhere,
admins are not forced to take a "submit/don't submit" or any other decision on the spot.

Therefore I believe that we can very easily integrate all of bug 1342 into an even more capable system. cool

Leaving that behind, and since it shouldn't be too difficult to complete phase two (no privacy/security issues up to that point), here are some thoughts about how to proceed:

Create two new database tables, mdl_health_settings (no need to clutter up $CFG with these, is there?) and mdl_health_errors (this is where we log everything). The exact schema needs a bit of careful thought.
Replace the PHP error_handler() with one of our own and put error_reporting(E_ALL) somewhere prominent. The idea behind this is that our error handler gets to log the messages, and also displays the error on screen as it does by default. It won't display E_NOTICEs if $CFG->debug == off, but it will log them. There shouldn't be any performance decrease, because we are doing extra work only when errors actually happen.
Prepare a proposal for an errorcode() function that has at least all the features discussed in bug 1342. Also, bug 1851 is mentioned and has another good idea.
Separate the health center in two parts: part one is for "configuration and security" (issues that are permanent) and part two for "runtime errors". The startup page will show a summary of items in both categories and allow the admin to delve deeper. Anyone is welcome to suggest things about this before I get round to writing code!

PS: Isn't it amazing how many solutions appear "by themselves" if you take a good long break from precise, demanding activities like coding?

Average of ratings: -