Good day,
for a database course format, when I upload content to a course it gets added to "data_content" table
When I did the upload (on the course page > settings > import > select file > click upload button) it was taking too long (over 1 hr for 100k rows of data)
so I created a new table in phpmyadmin and imported the CSV directly (takes about 10 mins)
I created a stand along page in a sub folder that looks at that new table and allows users to search the table using a form
In the case I take a vacation day, I don't want to give access to the server to some one that doesn't know what they are doing,
I wanted to create a page where the user is able to upload a csv file, it checks the file for quantity of columns, and then runs a "delete from {table}" sql, then imports the csv file into {table}
as far as the upload process I was following:
https://www.w3schools.com/php/php_file_upload.asp
https://moodledev.io/docs/apis/subsystems/external/files
I think i am trying to re invent the wheel from scratch, its there a way to re use with a "includes" for step 5 of this ?https://pacificu.libguides.com/moodle/fileupload#:~:text=Drag%20your%20file%20from%20your,to%20do%20with%20the%20file.4
I want to make sure I don't create a bigger problem down the line, is there anything else I need to take into consideration?
is there a better way of doing this ?
How do i give roles to users in a stand alone page? I can use if(is_siteadmin()) {} but what if i don't want to add another admin?
I'm still a novice for PhP / Moodle, and I understand just because it works now, doesn't mean its the best solution ... is there anything else I should be looking at?
It's very difficult to mix and match "plain" PHP with "Moodle"... so I think you've got 2 options for this.
One:
Given you've already defined a new table in your database that is nothing to do with Moodle, you can avoid anything to do with moodle and handle the filling of that table entirely via pure PHP.
One:
Given you've already defined a new table in your database that is nothing to do with Moodle, you can avoid anything to do with moodle and handle the filling of that table entirely via pure PHP.
Broadly speaking you'd need to:
- Create a PHP page with a file upload type.
- Have the PHP page stash the uploaded file (much along the lines of the w3schools example).
- Read the CSV data out,you can use fgetcsv()
- *but* you'll need to make sure you do the relevant error handling
- write each row into your table, but you'll have to use something like PDO to access your Moodle database (bypassing all of Moodle's DB API, which could be fine if you're just updating one non-moodle table).
Two:
Do it the "Moodle" way, which would involve writing some sort of plugin, which has it's own set overheads, but allows you to use the Moodle APIs to build forms. Your starting point for this would be https://moodledev.io/ and you may be wanting to look at something like a course report plugin.
This has the advantage that you get a lot of stuff to handle, including all your file-uploading & storage, security, data-sanitisation, validation, DB access, installation, csv parsing (look at csv_import_reader class for instance).
OK I thought of an option Three:
Work out why loading 100k of rows is taking so long!
You don't mention which course format you're using, which could be helpful to let other people see where you're fighting with
Thanks!, that's a great answer, this dev site would be a lot of help as well...
as for option 3, its a a single activity > database ... i
t took about 15 mins when I was on the other hosting site, but i have better hardware on the current one, same internet, using PHP 8.1 instead of 7.4, same Moodle version ...
same limits on php ini file
I do want to try to keep everything Moodle based as having stand along concepts work as a temporary solution .. but can be lost or if another programmer joins, they may not know where to look,
One thing I was thinking also was I can create a single activity page (this way it gets a course ID) ... and in the "/mod/data/view.php" I can add if ID = 123 ... then do the custom PHP things ...
That one does scare me, as if i mess something up it can mess it up for all the other views ...
I could do it as an include ... and that way it runs there, but lives on a "custom" folder
Thoughts?
as for option 3, its a a single activity > database ... i
t took about 15 mins when I was on the other hosting site, but i have better hardware on the current one, same internet, using PHP 8.1 instead of 7.4, same Moodle version ...
same limits on php ini file
I do want to try to keep everything Moodle based as having stand along concepts work as a temporary solution .. but can be lost or if another programmer joins, they may not know where to look,
One thing I was thinking also was I can create a single activity page (this way it gets a course ID) ... and in the "/mod/data/view.php" I can add if ID = 123 ... then do the custom PHP things ...
That one does scare me, as if i mess something up it can mess it up for all the other views ...
I could do it as an include ... and that way it runs there, but lives on a "custom" folder
Thoughts?