Unable to download large logs in Excel format

Unable to download large logs in Excel format

by Travis Penn -
Number of replies: 10

We are using Moodle 2.3.1+ and Postgres 8.4.  We had some Summer Courses which had a lot of activity.  When we try to view the logs, we can view them just fine.  But when we try to download them in excel or ODS or text, we get an error.   I'm sure this has something to do with the size of the logfile.  If we narrow our criteria we can usually download the file in excel, but we need a report of all of the activity not just for one day.

 

Here is the error we get in Chrome:

This webpage is not found. No webpage was found for the web address: http://moodle.nobl.k12.in.us/moodle/report/log/index.php?chooselog=1&showusers=0&showcourses=1&id=292&group=675&user=0&date=&modid=&modaction=view&logformat=downloadasexcel
Error 6 (net::ERR_FILE_NOT_FOUND): The file or directory could not be found.

 

Here is the error from the Postgresql Log:

2012-09-13 16:30:37 EDT ERROR: invalid input syntax for
integer: "View own submission status page." 2012-09-13 16:30:37 EDT
STATEMENT: SELECT name FROM mdl_assign WHERE id = $1

 

Here is the error from the Apache Error Log Debug:

[Thu Sep 13 16:30:37 2012] [error] [client] Default exception handler: Error reading from database
Debug: ERROR: invalid input syntax for integer: "View own submission
status page."\nSELECT name FROM mdl_assign WHERE id = $1\n[array (\n 0
=> 'View own submission status page.',\n)]\nError code:
dmlreadexception\n* line 407 of /lib/dml/moodle_database.php:
dml_read_exception thrown\n* line 239 of
/lib/dml/pgsql_native_moodle_database.php: call to
moodle_database->query_end()\n* line 708 of
/lib/dml/pgsql_native_moodle_database.php: call to
pgsql_native_moodle_database->query_end()\n* line 1346 of
/lib/dml/moodle_database.php: call to
pgsql_native_moodle_database->get_records_sql()\n* line 1419 of
/lib/dml/moodle_database.php: call to
moodle_database->get_record_sql()\n* line 1400 of
/lib/dml/moodle_database.php: call to
moodle_database->get_field_sql()\n* line 1379 of
/lib/dml/moodle_database.php: call to
moodle_database->get_field_select()\n* line 690 of /course/lib.php: call
to moodle_database->get_field()\n* line 178 of /report/log/index.php:
call to print_log_xls()\n, referer:
http://moodle.nobl.k12.in.us/moodle/report/log/index.php?id=292

 

I've attached a picture of the report that we are trying to export into excel.

Any ideas?  Is there a setting we should change in timeouts some place?  Is this just a bug with Moodle and Postgres?

php max_execution_time=600; max_input_time=60; memory limit is 512M 

 Thanks for any help.

Attachment Screen Shot 2012-09-13 at 4.34.42 PM.png
Average of ratings: -
In reply to Travis Penn

Re: Unable to download large logs in Excel format

by Mathew Powell -

Hi,

Just wondering if you have had any progress with this issue. We are experiencing similar but on a 2.1.5 version of Moodle.

 

Again if the logs are filtered down the files are generated without issue, so it looks to be related to the number of items getting pushed out to excel.

 

Many Thanks

Mathew

Average of ratings: -
In reply to Mathew Powell

Re: Unable to download large logs in Excel format

by Matias Pierrad -

I have the same problem.
With the difference that the file begins to download and then stops.
I have 72,000 records and the excel file contains only 32000.
I think there must be some timeout for it but do not know where edit.
If I find something on the subject, I notice.
Greetings from Argentina.
Matias. -

Average of ratings: -
In reply to Matias Pierrad

Re: Unable to download large logs in Excel format

by Dodji Nimrod -

Hello Matias and all,

Have you, please, found a solution to this problem? I am also having the same problem as stated above. And my guess is it's got to do with the size or number of items in the logs. But no solution so far!

I will be glad if anyone found the solution to this!

Many thanks

Dodji

 

Average of ratings: -
In reply to Dodji Nimrod

Re: Unable to download large logs in Excel format

by Perry Way -

In our case it's simply a matter of the web server timing out. You can adjust settings to make PHP live longer but that doesn't always work the way as intended. In our case I set PHP to never expire but the web server itself (IIS 7) doesn't want to wait that long and while you can also adjust that setting it doesn't always work as well (you can set the value as high as you want but IIS ignores everything over 30 seconds or 60 seconds I think. Believe me I have tried this before with some minor success but mostly failure).

If you really want this report data, perhaps it's proper to write a small wrapper to the function call in a php file you execute in a console instead of in a web page, then you can extend the execution time to as long as you want. I've had to do that on some really long running processes that integrate to other external systems.

Or, you could limit your data and then it will likely finish executing before the web server says things went too long.

Average of ratings: -
In reply to Perry Way

Re: Unable to download large logs in Excel format

by Dagwin Roelants -

Download in text format en copy paste from there to excel worked for me. (38.000 records)

Average of ratings: -
In reply to Dagwin Roelants

Re: Unable to download large logs in Excel format

by maark johnshon -

Try downloading the text format and copy paste in excel.it works.

www.hitech-cloud.com

Average of ratings: -
In reply to maark johnshon

Re: Unable to download large logs in Excel format

by Peiji Chi -

I face this problem too. I have to download around 60,000 logs, can not download in either TXT nor excel. Any other suggestion? 

Average of ratings: -
In reply to Peiji Chi

Re: Unable to download large logs in Excel format

by James D -

Same here. I can't download in any format.


Screen goes blank and nothing appears. No errors either.


Thanks,

James

Average of ratings: -
In reply to James D

Re: Unable to download large logs in Excel format

by Colin Fraser -
Picture of Documentation writers Picture of Testers

OK, this seems a well known issue, so has anyone placed an item into Moodle Tracker? There obviously seems a degree of urgency here, so the sooner it is done, the better, I suggest. I would do it, but I do not have an issue so have no errors to explain what might be happening. I don't have that many users per year.... 

Average of ratings: -
In reply to Travis Penn

Re: Unable to download large logs in Excel format

by Pramith Dayananda -

I've been having the same issue. This can go wrong in two places

* max_execution_time

* memory_limit

For the sake of one script increasing both from php.ini is not recommended. So I added following in the script

---------------------------------------

set_time_limit(###);

raise_memory_limit(###_###);

----------------------------------------

This works well for me while testing and did not found any side effects yet.



Average of ratings: -