UploadUsers (CSV) form refuses to load ...

UploadUsers (CSV) form refuses to load ...

by Ken Task -
Number of replies: 6
Picture of Particularly helpful Moodlers

After upgrading from 3.0.hghest to 3.1.highest (via git), an unusual situation.   Upgrade reported NO issues.   CentOS 6.highest, 8 Gig memory hosted on Rackspace (General Purpose offering).  PHP and MySQL versions acceptable for Mdl version.  Owerships/permissions are correct ... the 'Upload User Pictures' link comes from same directory but to a specific .php file.   UploadUsers goes to the index.php in /moodlecode/admin/tool/uploaduser/

Behavior noted ... after clicking on the link, an httpd service is launched and in a second or two, that one instance pegs the CPU at 100%.  In watching top, can see that same httpd process slip to 99.5% (or so) of CPU then back up to 100% CPU.   Does that until 'white screen of death' appears.   With Moodle error set to developer, nothing appears.   PHP settings for memory for a script to run set to 128MB, timeout set to 300 (was 90).  Max packets increased to almost it's max.   Can find nothing in servers logs that indicate an error.

Other odd behavior ... in logs ... for the admin user Moodle occassionally shows admin users last IP as IPv6.  Rackspace says one cannot disable the IPv6 assigned to server.

Have checked the uploaduser index.php script and in the top lines:

require('../../../config.php');
require_once($CFG->libdir.'/adminlib.php');
require_once($CFG->libdir.'/csvlib.class.php');
require_once($CFG->dirroot.'/user/profile/lib.php');
require_once($CFG->dirroot.'/user/lib.php');
require_once($CFG->dirroot.'/group/lib.php');
require_once($CFG->dirroot.'/cohort/lib.php');
require_once('locallib.php');
require_once('user_form.php');

$iid         = optional_param('iid', '', PARAM_INT);
$previewrows = optional_param('previewrows', 10, PARAM_INT);

core_php_time_limit::raise(60*60); // 1 hour should be enough
raise_memory_limit(MEMORY_HUGE);

All files required are present and readable, but that the heck does that last line raise the memory to?

Site is not using groups nor cohorts.

I'd write this up for Tracker, but 3.1 (even though 'long term support') is for security fixes only now.

Anyone see anything like this?

'spirit of sharing', Ken


Average of ratings: -
In reply to Ken Task

Re: UploadUsers (CSV) form refuses to load ...

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Can you upload any files (i.e. anywhere else in Moodle)?

Average of ratings: -
In reply to Howard Miller

Re: UploadUsers (CSV) form refuses to load ...

by Ken Task -
Picture of Particularly helpful Moodlers

Thanks for reply.   That's the funny part about this ... yes, I can upload files - drag and drop as well as choosing from a file system repo (restoring backups).

That one link ... uploaduser (CSV)  ... form doesn't generate at all - not even partially.

May have discovered a little bug ... in the Moodle Admin UI, one cannot use  online YUI libraries with https sites - says it's not compatible with https.   Even if one attempts to check the box, the form comes back 'unchecked'.    I set that in config.php file (correctly).   I see in the admin area for AJax

https://site/admin/settings.php?section=ajax

'defined in config'.

But, in looking at mdl_config table in the DB, 'useexternalyui' variable was still set to '1' ... meaning on.  I've set that manually in the DB now also.

YUI combo loading is still set to yes ... ie, 1 or use.

Prior to the upgrade from 3.0 to 3.1, I don't re-call what that setting was but the site has been https since somewhere late version 2.9, if I re-all correctly.   It has been working.

Memory according to top:

Mem:   8053668k total,  7897496k used,   156172k free

Still ... the form does not render, top shows a single http process peg the CPU at 100% ...

[root@www cli]# php cfg.php |grep yui
useexternalyui    0
yuicomboloading    1
yui2version    2.9.0
yui3version    3.17.2
yuipatchlevel    0

Should combo loading be turned off as well?

This is very crazy!!!!

Would appreciate any ideas!!!!


'spirit of sharing', Ken


Average of ratings: -
In reply to Ken Task

Re: UploadUsers (CSV) form refuses to load ...

by Dimitar Ivanov -

Code says MEMORY_HUGE means 2GB (https://github.com/moodle/moodle/blob/MOODLE_31_STABLE/lib/setuplib.php#L1251), but that's not something that changed between 3.0 and 3.1. 


I would start with the simple things like:

 - purge all caches, upgrade should do that, but sometimes it doesn't so manual purge could help

 - do a clean install in another location (sub dir/virtualhost) and see if it behaves differently

 - start commenting out lines from that index.php file (from top to bottom) and see when it breaks

   * shortcut option, if you think that the memory raise is the issue comment that out first, if there is not enough memory PHP will tell you


EDIT:

Now that I looked at the code again, I see another possibility, if you have $CFG->extramemorylimit set to something bigger than 2G, then Moodle will use that, do you by chance happen to have that config option set in your config.php?


Average of ratings: -
In reply to Dimitar Ivanov

Re: UploadUsers (CSV) form refuses to load ...

by Ken Task -
Picture of Particularly helpful Moodlers

Thanks for response and suggstions.

2 Gig huh.   Well, according to free on said server:

             total       used       free     shared    buffers     cached
Mem:       8053668    3315116    4738552     102168      70036     337824

Yep, tried all other suggestions ... purged cache via Moodle UI and manually.

Tried to setup a 3.2 in a subdirectory of the site ... would have loved to do a virtual, but that would mean DNS additions (yes, know I could cheat with local host file but I want the owner to be able to see the dev site ... he can't do what needs to be done on Windows and doesn't see why that's necessary anyway! :\)

So subdir install.   Command line via git, no issues installing, but, it did push memory usage higher ... thus leaving only about 1.3 Gig free mem available.   That turned out to be affecting production site performance so had to remove it.

I still see what I am thinking is a catch 22 ...

In config of YUI.

Use online libraries is now set to no ... by default.

Instead of using local files, use online files available on Yahoo‘s servers. WARNING: This requires an internet connection, or no AJAX will work on your site. This setting is not compatible with sites using https.

Site is https ... no issues with certificates.   Seems kinda strange to me as just about all other resources ... like google, etc. would prefer the 'mooch' to use https .. and setup keys, secrets, etc. to be able to use.  But, since YUI library no long being updated, good to move away from.

Yet, the other setting:

Combo loading ... is configurable.

Comment for that says:

This options enables combined file loading optimisation for YUI libraries. This setting should be enabled on production sites for performance reasons.

To me, even if it means performance suffers, I'd rather it work.   So I've set that both ways ... with the same results.

So ... ????

The only other thing left is config of apache.   The site isn't a large one ... a little under 3000 users, not serving anything I'd consider heavy ... but did have an issue with timed quizzes ... student submitted but grade book showed 'In progress' (that's yet another discovery after upgrading).   Did set max_packets_allowed higher than before, which seem fix that issue.

Apache config never has needed much tweaking ... cept for when moving from http to https.

Am about to propose what you've suggested ... fresh installation via git of 3.1.highest, copy over the plugins ... there's only two ... Customized Certs and Customizable Reports (the latter hardly used at all).

The Customized certs, however, does appear to use YUI libraries - dang ... another catch 22!!!!

I see by your example of forcing an error it was done with a wordpress.   Ever done that with a Moodle?

I did comment out that line for huge memory ... which still did not render *any* error ... not in Moodle and not from PHP/Apache ... not even in php/apache/messages logs of the server.

Unfortunately, Rackspace, like all hosting providers, don't go into apps and ticket repoonses naturally say consult Moodle Community Forums.   Didn't have the heart to respond by telling them about my relationship with Community Forums. ;)

THanks again for suggestions ... and for listening. :|

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: UploadUsers (CSV) form refuses to load ...

by Dimitar Ivanov -

I see by your example of forcing an error it was done with a wordpress.   Ever done that with a Moodle?

- No, I have not tried it on Moodle, it's a generic PHP error, that should be reproducible everywhere but Moodle might be "swallowing" it to show it's own message. The Wordpress example is the one I found trying to give an example on what it looks like.


Another thing that you could try is disabling the Theme caching, that is "Enable Theme designer mode". That way Moodle will load each CSS and JS file separately and will hopefully help you see where the error comes from.



Average of ratings: -
In reply to Dimitar Ivanov

Re: UploadUsers (CSV) form refuses to load ... RESOLVED

by Ken Task -
Picture of Particularly helpful Moodlers

First, thanks guys for responding and offering suggestions.

Am soooooo angry at self I could spit!!!!   Kept looking in the wrong spot ... Moodle runs under PHP so it actually tells Apache what to do and how to do it (so to speak) ... duh!   Started checking into PHP mods loaded ... on the server that worked, no Zend xdebug, no apc nor apcu ... only ONE Zend loaded.

On the server that did not work ... had that run away single apache process that pegged the CPU any time one accessed the uploaduser form ... had not only Zend xdebug but apc and apcu + what appeared to be yet aother Zend Opcache.  I did think that possible ... but ... DANG machine smarter than I am?

The really screwball thing ... it worked in Moodle version 3.0.highest!!!!

I share not only solutions to issues, but am also living proof of how one can shoot themselves in the foot!   This offered in hopes that someone who has a similar problem changes their focus. ;)

Am going to get a cold beer ... high 90's in SA today ... 'couch potatoe' the rest of the evening.

Tomorrow will be a better day! (it better be!!!!) :|

'spirit of sharing', Ken


Average of ratings: Useful (2)