Bulk / automated course creation with phpMyAdmin

Bulk / automated course creation with phpMyAdmin

by John Molohan -
Number of replies: 80
Hello all,

I'm trying to automate the creation of a couple of thousand courses for our moodle site. I'm using phpMyAdmin with some sucess. I can use a text file to insert multiple courses but I need some help and clarification on some of the fields (caveat, I'm new to sql, php, moodle smile ).

At the moment we use these Column names id,category,sortorder,fullname,shortname,idnumber,summary,format,showgrades,modinfo,blockinfo,newsitems,teacher,teachers,student,students,guest,startdate

And our text file looks like this

'','2','999','abc0094','A N Other test No 094','NCC-F-AN094','NCC-F-AN094','This is another 0094 test','topics','1','','','5','Lecturer','Lecturers','Student','Students','','10'

This is working.

The id number is generated by myslq when we use a blank field entry ie ''.

The first problem I have is with the sortorder, with a blank field moodle does not autogenerate this and fails the import. If I generate random or predefined sortorder values how will these work when new courses are added through moodle? How does moodle itself generate these sortorders? Could I get moodle to auto create these?

The second question is can someone clarify the modinfo and blockinfo entries, what values they take and what data that corresponds to. I'm pushed for time on this project so I'm sorry if this has been covered elsewhere but I've searched moodle.org and done my googling but turned up nothing.

Any help appreciated,

John Molohan

Moodle version 1.4.4

Average of ratings: -
In reply to John Molohan

Re: Bulk / automated course creation with phpMyAdmin

by Scott Krajewski -
With our mass course creation, similar to yours but using direct db queries from an external machine, we ignore the modinfo and let moodle fill that in.  So far so good for us.  But the blockinfo determines what blocks you want to show up and where by default -- very handy!

Remember each block has a unique id in mdl_blocks.  In ours activity_modules is id=1, course_list is id=5 and online_users is id=9 for example.  Say you want the first two on the left and the third on the right for all of your autogenerated courses.  In that case, blockinfo = "1,5:9" .  block ids to the left of the colon show up on the left, the right on the right.

I'm not sure about the sortorder.  I wound up finding the highest current value in the table and incrementing by 1 for each course.  I suspect these are updated when you sort courses when browsing your categories.

--S
Average of ratings: -
In reply to Scott Krajewski

Re: Bulk / automated course creation with phpMyAdmin

by John Molohan -
Thanks for the reply. We've switched to v1.5 now which is slightly different, the block info is incorporated into modinfo. I've created a course with the blocks as wanted then copy this modinfo from mdl_course into a csv to test an import, the info is taken in, courses created no errors, I can see the correct modinfo populated in the table but when I actually go into the new course there are no blocks. I've had too many late nights recently so I've probably missed something obvious, I'll check tomorrow.

John
Average of ratings: -
In reply to John Molohan

Re: Bulk / automated course creation with phpMyAdmin

by Scott Krajewski -
It looks like 1.5 has a new way of configuring blocks on pages.  It's pretty slick.

The block info is now stored in a new table called mdl_block_instance with 9 fields:

id = unique table index
blockid  = the unique id of the block from mdl_block
pageid = essentially the unique course id where the block is
pagetype = I think this is for blocks showing up on a course page vs module page.  All of mine say course-view.
position = left (l) or right (r) ?
weight = 3 is lower on the page than a 2.  It's an ordering, 0 is a the top of the page side.
visible = 1 -> yes, 0 -> no
configdata = must for be for blocks that need to save some configuration data.

I found that creating a course in 1.5 via direct database SQL as I did in 1.4 yields a course with no blocks.  Inserting rows to mdl_block_instance adds the blocks I want to the course in the location I want.

Nicely modular.  It was a good idea to strip the block display out of the course data since modules can have blocks now.

Hopefully this will help.  Now I have to update my routines to do this.

--S

Average of ratings: -
In reply to Scott Krajewski

Re: Bulk / automated course creation with phpMyAdmin

by denis cahalane -
I'm glad mass creation of courses is working for others - it has stopped working for me since I upgraded. I'm on 1.52 with mysql. I'm creating the block_instance entries but they're just being ignored.

For example, I set up a new empty instance and created one course. Then I added 8 block_instance entries to match it. Then I entered the course and found no blocks, so I added the Participants block manually. That adds one line to mdl_block_instance identical to one of the lines I had already added. The first select below shows the blocks before the manual addition, the second afterwards.



mysql> select blockid,pageid,pagetype,position,weight,visible from mdl_block_instance;
+---------+--------+-------------+----------+--------+---------+
| blockid | pageid | pagetype    | position | weight | visible |
+---------+--------+-------------+----------+--------+---------+
|      19 |      1 | course-view | l        |      0 |       1 |
|       2 |      1 | course-view | l        |      1 |       1 |
|       5 |      1 | course-view | l        |      2 |       1 |
|       6 |      1 | course-view | r        |      0 |       1 |
|       3 |      1 | course-view | r        |      1 |       1 |
|      13 |      2 | course_view | l        |      0 |       1 |
|       1 |      2 | course_view | l        |      1 |       1 |
|      17 |      2 | course_view | l        |      2 |       1 |
|       2 |      2 | course_view | l        |      3 |       1 |
|       5 |      2 | course_view | l        |      4 |       1 |
|      11 |      2 | course_view | r        |      0 |       1 |
|       4 |      2 | course_view | r        |      1 |       1 |
|      15 |      2 | course_view | r        |      2 |       1 |
+---------+--------+-------------+----------+--------+---------+
13 rows in set (0.00 sec)

mysql> select blockid,pageid,pagetype,position,weight,visible from mdl_block_instance;
+---------+--------+-------------+----------+--------+---------+
| blockid | pageid | pagetype    | position | weight | visible |
+---------+--------+-------------+----------+--------+---------+
|      19 |      1 | course-view | l        |      0 |       1 |
|       2 |      1 | course-view | l        |      1 |       1 |
|       5 |      1 | course-view | l        |      2 |       1 |
|       6 |      1 | course-view | r        |      0 |       1 |
|       3 |      1 | course-view | r        |      1 |       1 |
|      13 |      2 | course_view | l        |      0 |       1 |
|       1 |      2 | course_view | l        |      1 |       1 |
|      17 |      2 | course_view | l        |      2 |       1 |
|       2 |      2 | course_view | l        |      3 |       1 |
|       5 |      2 | course_view | l        |      4 |       1 |
|      11 |      2 | course_view | r        |      0 |       1 |
|       4 |      2 | course_view | r        |      1 |       1 |
|      15 |      2 | course_view | r        |      2 |       1 |
|      13 |      2 | course-view | r        |      0 |       1 |
+---------+--------+-------------+----------+--------+---------+
14 rows in set (0.00 sec)

mysql> select id, fullname from mdl_course;
+----+--------------------------+
| id | fullname                 |
+----+--------------------------+
|  1 | fdb                      |
|  2 | Test course for John Doe |
+----+--------------------------+


Average of ratings: -
In reply to denis cahalane

Re: Bulk / automated course creation with phpMyAdmin

by scott hallman -

Apologies I'm not a programmer but the E-learning Co-ordinator who had to load 900 courses manually last term and who doesn't want to have to repeat that this term. I have no idea about php script so was wondering if you know of a step by step guide that I can follow. Hoping I can have a holiday this break instead of spending it loading courses to moodle. All I have is an excel spreadsheet.

thanks for your time

Scott

Average of ratings: -
In reply to Scott Krajewski

Re: Bulk / automated course creation with phpMyAdmin

by Gavin McCullagh -
> With our mass course creation, similar to yours but using direct db
> queries from an external machine, we ignore the modinfo and let
> moodle fill that in.

Does anyone know of any developer stuff lying around for this such as perl modules or whatever?  I'm working with John here to do some bulk imports.  Just checking it doesn't already exist before I go writing it myself.

Gavin
Average of ratings: -
In reply to Gavin McCullagh

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -
If anyone's interested, here's the guts of the script we use at the start of each year to create courses. It had to be optimised a bit, as using Moodle's in-built functions to create 1000s of courses takes a long time (now down to about a minute as opposed to half an hour!). More importantly it creates all the courses "properly" and Moodle seems happy with them 12 months down the line.

It loads the courses to be created into $bulkcourses, which in this instance comes from another database, but could be loaded from a CSV file or whatever.

Of course make a backup before doing anything to your database smile
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Konu Ullah -

Hi Rory Im trying to run your php script but Im getting this error

E:\moodle>createcourse.php
Content-type: text/html
X-Powered-By: PHP/4.3.9

<br />
<b>Warning</b>:  main(../config.php): failed to open stream: No such file or dir
ectory in <b>E:\moodle\createcourse.php</b> on line <b>32</b><br />
<br />
<b>Fatal error</b>:  main(): Failed opening required '../config.php' (include_pa
th='.;c:\php4\pear') in <b>E:\moodle\createcourse.php</b> on line <b>32</b><br /
>

My php knowledge is very limited so any help appreciated.

TIA.

Average of ratings: -
In reply to Konu Ullah

Re: Bulk / automated course creation with phpMyAdmin

by Scott Krajewski -
It looks like createcourse.php assumes it is in a subdirectory of your moodle directory (that's why it's looking for config.php in ../config.php).  Either change the "require_once" paths or move it to a subdirectory of your moodle install.

Also, make sure  $bulkcourses is set to fill with your course data.

This is a nice little script.  It would be great to have an "upload CSV" interface for it in the admin screen, no?

--Scott
Average of ratings: -
In reply to Scott Krajewski

Re: Bulk / automated course creation with phpMyAdmin

by Konu Ullah -

Hi Scott, thank you for that an upload interface would be so useful .  I'm sure their are many users who put on more that 10-20 courses manualy. 

Now I'm getting this output.  If someone can kindly guide me on how I change the script to look at my csv file that would be a great help.

E:\moodle\admin>createcourse.php
$CFG->aspellpath in config.php ("C:\Program Files\Aspell\bin\aspell.exe") overri
des database setting ()
Content-type: text/html
X-Powered-By: PHP/4.3.9
Set-Cookie: MoodleSession=62d70245db17e8796d4dd6fdf6059904; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: MoodleSessionTest=vGS3J81TG8; path=/

<div class="notifyproblem" align="center">Warning: Could not find any of these w
eb server variables: $REQUEST_URI, $PHP_SELF, $SCRIPT_NAME or $URL</div><br />
<meta http-equiv="refresh" content="0; url=http://waes-elearn.waes.ac.uk/moodle/
login/index.php" /><script type="text/javascript">
<!--
location.replace('http://waes-elearn.waes.ac.uk/moodle/login/index.php');
//-->
</script>

Thanks

Average of ratings: -
In reply to Konu Ullah

Re: Bulk / automated course creation with phpMyAdmin

by Scott Krajewski -
Konu, try calling it via the web instead of the command line to avoid those errors. I suspect it wants to be run that way. http://waes-elearn.waes.ac.uk/moodle/admin/createcourse.php

Maybe someone could use the code in uploaduser.php to fill the array here? smile Or there's always the code for flatfile enrolment to borrow from. Sorry but it won't be me though as we're entering our final month of classes. But it should be easy enough to adapt the flatfile code to this.....

--Scott
Average of ratings: -
In reply to Scott Krajewski

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -
Thanks Scott - admin/uploaduser.php seems to be easy to modify, I'll give it a go.

BUT, why the strange aversion in Moodle to the fgetcsv and fputcsv functions??? They've been available since php3, are much faster (and easier) than any Perl-like implode/explode trickery, and will handle quoted fields unlike the former, e.g. in typical CSV from a spreadsheet. Anyway. smile
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -
OK here we go put it in /admin and it should work, though it needs extensive testing. Perhaps someone could add in cron support as well. Woah, is that the time... smile

BTW is anyone else noticing wierd things with the moodle.org mimetypes?
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -
Improved version, and an example CSV file. smile
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Konu Ullah -
Hey Rory, that is superb, many thanks.
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -
New ability to use a "category path" slash delimited string to specify the new course location, and it will create categories with those names as necessary if they don't exist.

Now you can design your whole Moodle site's structure in a spreadsheet, which is far easier and quicker, especially if you have 1000+ courses and categories! smile

Tarball contains updated version, new example CSV and instructions.
Average of ratings: Useful (1)
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Pablo Etcheverry -
Picture of Core developers
Rory, this is a great addition. Do you think it could be modified to create not only empty courses but also full courses starting from a ZIP backup or template?
Cheers,
Pablo
Average of ratings: -
In reply to Pablo Etcheverry

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -
Nice idea - It should be simple enough to insert a call to restore_execute() in backup/restorelib.php at the end of the course creation loop, defining RESTORE_SILENTLY = 1 and using the import course data option, which would also act as a "template".

I'll do this when I have time, or if anyone else wants to please go ahead smile.
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Nivaldo Vasconcelos -
Hi Rory,

great !!
I would like to do this. Will be faster if you can help me.
Please, send more details about this. Files, line number, etc.
I can implement yours sugestions, test and feedback.

Best regards,
Nivaldo
Average of ratings: -
In reply to Nivaldo Vasconcelos

Re: Bulk / automated course creation with phpMyAdmin

by Mark Hilliard -
Nivaldo,
I am VERY interested in a bulk course creation tool that would utilize a restore of a template course as discussed by you in this thread.

Did you ever complete a working prototype of this script to take a course zip file, restore it to a new course with an assigned new teacher?

Thanks for any info on status you might be able to share.

mark hilliard
Minneapolis, MN
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by scott hallman -

Thank you for your solution to our delema. You have enabled me to have a break this holiday instead of logging 900 courses to moodle. This as you can imagine has made me quite happy and enabled me to get on with more exciting things like adding special effects to my short films.

So lots of thanks

Scott

Average of ratings: -
In reply to scott hallman

Re: Bulk / automated course creation with phpMyAdmin

by adam buxton -
Just the thing i was looking for thank you very much to rory and all above..

Average of ratings: -
In reply to adam buxton

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -

No problem, nice to know other people are finding it useful as well  smile.

In that case here's my latest version, complete with the ability to specify topic headings, and enrol teachers (though not students as the existing enrolment plugins should be used), ready for the start of the next academic year!

I've also included a script for printing a collapsible heirarchical view of your Moodle site down to topic level which someone might find useful as well, especially for end of term reports.

Grrr, 10KB limit, now at http://rory.allford.net/uploadcourse.tar.gz.

Perhaps someone could put this into CVS, and possibly translate it?

Average of ratings: Useful (3)
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Alan Hicks -
Great work, thanks Rory.

Can you point me in the right direction to get this code running as a CRON job. I want to run this every night incase there is a change in course information.

Any info would be great. I know that there is a CRON php file in moodle but am struggling to work out how I could modify this to help.

Many Thanks,
Alan.
Average of ratings: -
In reply to Alan Hicks

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -
Hi Alan,
I intended it really as a one-off manual admin tool, so it won't easily run as a cron job as it is.

Refactoring it into a plugin which would be necessary to run properly within Moodle cron would require considerable of work (I'm not even sure what type of plugin would be suitable), so it would be simpler to run it separately.  Cronifying the script would involve changing it so that output is logged in case of errors e.g. Moodle's mtrace(), and the input file would need to be changed from an upload object to a path.

This would be an interesting project, especially with the ability to change and delete existing courses as well, and perhaps supporting different data sources as the enrolment plugins do.

However I'm quite busy at the moment, and personally I wouldn't need this functionality, but do forge ahead yourself smile.
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Alan Hicks -
Thanks for the quick response. I have no PHP coding experiance but from looking at the existing cron.php file I'm going to give it ago.

If I get anywhere I will post the results here.

Wish me luck big grin
Average of ratings: -
In reply to Alan Hicks

Re: Bulk / automated course creation with phpMyAdmin

by Greg Rodenhiser -
Dumb question....  Once I use this feed to generate all my courses....  How do I purge them all out?  I used this to import a feed of about 700+ courses for testing, now I'd like to remove the 700+ courses and start again.  I'd like to be able to do this after each semester.  Thanks!!
Average of ratings: -
In reply to Greg Rodenhiser

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -
Hi Greg,
The tool doesn't have a bulk delete facility yet, but it would be fairly simple to implement it (running against the same CSV file and calling delete_course()), and I do intend to when I get a chance smile.

Better in my opinion, however, is to make a backup immediately before you run the script: if you find mistakes, you can always revert and try again.

Actually on our site we've never had to delete courses; at the end of each year they are archived off into a category rather than deleted, the benefit being that students can access their old course for reference, and teachers can recycle material and reference all the old logs and grades.
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Greg Rodenhiser -
Sounds like a decent way to handle old courses (simply don't make them visible).  Now one more question...  What flag (heading) can I put in my CSV file to disable enrollment in the course?  I do not want any students to be able to enroll in the courses created by this script (as I will be using Moodle's built-in flat file enrollment process).  Thanks, I really appreciate it....
Average of ratings: -
In reply to Greg Rodenhiser

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -
Currently, Moodle does not have per-course enrolment options: to prevent students manually enrolling themselves into any courses, make sure the enrol_allowinternal checkbox is unchecked on the Flat File enrolment page. Students will then get the message "Manual enrolments are disabled" when they try and enrol themselves.

We've set our courses up to completely prevent guest/read-only access as well for non-enrolled students; this can be done by setting the value of "guest" to 0 in your CSV file for all courses. They'll then be refused access with "This course does not allow public access".
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by poll yakadi -
I download and sucessfully installed this uploadcourses script to moodle and created few large course categories.
Can anybody guide me how to use this script to create only the categories and subcategories without bothering about large amount of courses inside those categories.
What I realise is that it checkes for shortname of course in csv, and if it is not available it gives a error.
Average of ratings: -
In reply to poll yakadi

Re: Bulk / automated course creation with phpMyAdmin

by Rory Allford -

It was originally intended as a course creation tool, though it's simple enough to change it (and I will do when I have a chance) so that you can use it to create categories only (i.e. by only having the category field in your CSV file). However it seems a little odd to want categories with no courses in them!

In the meantime you can create the categories by specifying each unique category path with some course details, and then deleting the course.

Average of ratings: -
In reply to Rory Allford

New question RE: Example CSV

by Jeff Herman -

What is the "Marker" field in example_teachersenroll.csv?  Does this correspond to  the option "Force language"?

Average of ratings: -
In reply to Jeff Herman

Re: New question RE: Example CSV

by Red Morris -
This is fantastic! Just what I needed, and worthy on including in the base Moodle build in my opinion
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Manuel de la Torre -
Hello,

I need your help... Does any body knows how to install the uploadcourse plugin in Moodle 1.7, the admin structure has changes and the directions provided are for previous versions.

Thanks,

Manuel.
Average of ratings: -
In reply to Manuel de la Torre

Re: Bulk / automated course creation with phpMyAdmin

by Jeff Church -

I would like to second Manuel's question. Does anyone have a fix for running this in 1.7?

Average of ratings: -
In reply to Jeff Church

Re: Bulk / automated course creation with phpMyAdmin

by Rodrigo Oliveira -
Hi folks,

I've been working with this functionalitie in 1.8 beta+. First, it works well but needs some tweaks. One thing that I can't fix is the creation of a new option in the admin menu tree. If someone has achieved this in 1.8, please, share with us.

I've also noted a few things that I would like to bring to discussion.
First, in 1.8 beta +, when I create a course (manualy), the plataform change at least 3 tables: mdl_block_instance, mdl_course and mdl_context.
The mdl_block_instance contains information about what blocks will appear in the course page.
The mdl_course contains, obviously, information about the course (properties).
The mdl_context gains a new record that indicates what type of context is the new instance_id, that is, when I create a course, this table will have a new record with contextlevel 50 (meaning that this is a course) and with instanceid equal to the new course courseid.

I dont' know what happened in the earlier versions but seems to me that al this changes are a little bit important.
Is anyone working with this version ant tried to implement this course creation technique?

Average of ratings: -
In reply to Jeff Church

Re: Bulk / automated course creation with phpMyAdmin

by Red Morris -

I am using 1.7.1 and haven't had any problems with the few courses I've created so far.

I placed the php file in the admin folder and navigated to moodle/admin/uploadcourse.php.

Average of ratings: -
In reply to Manuel de la Torre

Re: Bulk / automated course creation with phpMyAdmin

by Erich Schubert -
I had also a problem with uploadcourse. I found that the creation/assigning of a teacher had changed (in case of the new roles). The creation part of a categorie/course works.
To solve this problem some lines of the sourcecode must be rewritten, but i had no time to do this. So I worked with following solution:

  • create the categories/courses with a minimal "course.csv" und bulkupload (see sample testcourses.csv)
  • assign the teachers with the integrated upload users


The sample testcourses.csv
fullname, shortname, category, format, visible, password, guest, topic1
Course_1, c_01, /j2007/test, topics, 1, test, 0, Course
Course_2, c_02, /j2007/test, topics, 1, test, 0, Course


The sample testuser.csv
username, password, firstname, lastname, email, course1, type1
user1, pass1, first, last1, 01.user@nobody.nix, c_01, 2
user2, pass2, first, last2, 02.user@nobody.nix, c_02, 2

This solution works alos with 1.8+


Average of ratings: -
In reply to Erich Schubert

Re: Bulk / automated course creation with phpMyAdmin

by Red Morris -

I am having an issue with using this on 1.7 too. This may be the same roles related one Erich refers to.

When I put the unique logon ID into the CSV it errors saying that it found no results. If you go to the Browse All Users page and search on an ID you get 0 results there too. So it seems that the one field we can be sure is unique cannot be used! Email fields are also unique, but not predictable. We allow staff two email addresses, initialsurname@... and forename.surname@..., or they may choose an external email for all we know!

I'm a competent programmer, and new to PHP, but Moodle is still hard for me to get into and be sure I understand it, so making changes myself isn't 100%.

EDIT: I've noticed that the userexists_ex function doesn't actually look for the id. I will try to change this so it does. I think it's an easier unique identifier to work with than the ID number

Average of ratings: -
In reply to Red Morris

Re: Bulk / automated course creation with phpMyAdmin

by Red Morris -

Well, I have made progress. I have modified the main page to accept usernames and verify them. With the data coming from the same field regardless of what it is, I've had to use the detection of a space to determine if it's a name or username.

The problem now is that the script is set up to add to pre-Roles-Moodle tables. Can anyone help me work out what the INSERT at line 500 needs to be for it to work with 1.7+? A little table detection and it should be able to work for both.

Average of ratings: -
In reply to Red Morris

Re: Bulk / automated course creation with phpMyAdmin

by Garret Gengler -

Hello... I'm taking a look at this.. for Spring semester, we used an in-house script to create courses.  But that was written for the 1.5.x data model, and would need a lot of rework.

The 1.7 roles stuff is tricky... lemme see if I can figure out what's going on in that section of code.

-Garret
 garretg@uiuc.edu

Average of ratings: -
In reply to Garret Gengler

Re: Bulk / automated course creation with phpMyAdmin

by Garret Gengler -

Okay, so looking at the code... The funny thing is, in the Roles model, there is not really a "Teacher" for a course anymore.   The teachers of a course are any users who have the capabilities they need to teach, in the context of that course.   The role that grants them the needed capabilities is arbitrary... it could named anything, and have any id number.

So I think the cleanest approach will be to change the "teacher" parameters to something more generic, like:

users[1,2,3]
roles[1,2,3]

As an example.. for a given course (i.e., on one line of the input file

users[1132,1562,1121], roles[3,3,3]

... 1132, 1562,1121 are the ids of rows in the users table, and 3 is the id of the Teacher role.

The code could be smart enough to let you pass a username instead, or email address (assuming it's unique in the db), and also to let you pass a role by name.  And it should use the new assign_role routines to make the role assignments, not talk straight to the db tables.

I have a little time today and tomorrow, and I need this very much.. so I'm going to see if I can rig this up.

Average of ratings: -
In reply to Garret Gengler

Re: Bulk / automated course creation with phpMyAdmin

by A. T. Wyatt -
We need it too! I am over here cheering for you!

atw
Average of ratings: -
In reply to Garret Gengler

Re: Bulk / automated course creation with phpMyAdmin

by Red Morris -
It's the contextid field that I need to work out. Where does this id come from, or how is it calculated? Once I've got that I think I'm away.
Average of ratings: -
In reply to Red Morris

Re: Bulk / automated course creation with phpMyAdmin

by Garret Gengler -

The contextid comes from the id column of mdl_context.  Basically, each context level in the site has a record in that table... think of a context level as a level at which a particular set of permissions applies.   Each course has a record in the mdl_context table, with contextlevel=50 (it's a constant meaning course level) and instanceid=the course id.

Hmm... I thought you'd just be able to call the new role_assign routine, as is done in the now-deprecated add_teacher function.  But to call that, you need to pass in a contextid. 

I wonder now if this is worthwhile... maybe the uploadcourses script should not add teachers... enrollment plugins offer many elegant ways to associate users with a course, and to create the users if need be.

Come to think of it, that would work fine for my purposes.   As someone else described above.. I can use uploadcourses.php to create the bare courses, then uploaduser.php to create and/or associate teachers with the courses.

Average of ratings: -
In reply to Red Morris

Re: Bulk / automated course creation with phpMyAdmin

by Garret Gengler -

Alright, after looking into this a bit more...

After you add a course to to the mdl_course table, you can ask for the course context object with this code:

$context = get_context_instance(CONTEXT_COURSE, $newcourseid);

Then just use $context->id  when you need the context id... say as an input to assign_roles().

If the course context entry doesn't exist yet in the mdl_context table, it will be created automatically.   (That must be why this uploadcourse.php script works for people in moodle 1.7+ despite containing no roles-related code.)

/course/edit.php contains very similar code to what this script is doing... it likely has a few more things that should be done.

Average of ratings: -
In reply to Garret Gengler

Re: Bulk / automated course creation with phpMyAdmin

by Red Morris -

Well, I've started again and got to the point where I can set a network id in the teacher_account field and it will enrol that person as a teacher. Yay!

However, all courses must have the same number of teachers. If there's a teacher_accountX field then every course must fill in details for it or it will fail validation. I think this must be an existing limitation of the page, so I'm wondering if someone else has already cleared this hurdle. And if so, can I have the segment of code please?

Average of ratings: -
In reply to Red Morris

Re: Bulk / automated course creation with phpMyAdmin

by Yann FOUCAUD -
Hello.
Is it possible to have a version which allows the creation of several courses under only one category? Which lines to change for that? Thank you.

example:
course 1, course1, /category1,...
course 2, cours2, /category2...
will ok, whereas
course 1, cours1, /category1, ...
course 2, cours2, /category1...
allow creation only first course. Thank you.
Average of ratings: -
In reply to Yann FOUCAUD

Re: Bulk / automated course creation with phpMyAdmin

by Erich Schubert -
Use a csv-file with following structure

fullname, shortname, category, format, visible, password, guest, topic1
Course_1, c_01, /j2007/test, topics, 1, test, 0, Course
Course_2, c_02, /j2007/test, topics, 1, test, 0, Course

and do an upload with the latest version of bulkupload.

Average of ratings: -
In reply to Erich Schubert

Re: Bulk / automated course creation with phpMyAdmin

by Lars Jensen -
Hi Erich,

Thanks for your answers. This procedure works quite well. The most annoying problem I have with it is that the default blocks that are present on the main course page when courses are created manually, are not present when courses are created with the csv upload. Did you solve manage to solve this problem? Also, what table in the database tracks what blocks are present in a course?

Thanks so much for your help,
Lars.
Average of ratings: -
In reply to Erich Schubert

Re: Bulk / automated course creation with phpMyAdmin

by Lars Jensen -
Hi Erich,

Thanks for posting this. I have a copule of questions:
  1. Is the testcourses.csv to be uploaded directly into the mdl_course table in the database?
  2. What happens to the fields in this table which are not defined?
  3. It looks like your format for the category doesn't agree with the one in mdl_course, where the category is listed by an id number. The actual name of the category seems to be defined in mdl_course_categories. Your testcourses.csv file looks like it uses the acyual name of the category.
  4. Is the password in testcourses.csv the same as the enrollment key?
Thanks for your help.

Lars.
Average of ratings: -
In reply to Lars Jensen

Re: Bulk / automated course creation with phpMyAdmin

by Erich Schubert -
Hi Lars,

Following answeres:

  1. The testcourses.csv ist directly loadet to the mdl_course table.
  2. The not defined fields stay empty or where filled with defaults.
  3. That is correct, my version uses the actual name of the category (internally the actual name is written to mdl_course_categories, and the id is read back)
  4. The password ist the enrollment key.

If you want i can send you my version of bulkupload. I made a manual too, but it's only in german. I think ist is usable because there are screendumps where you can see what to do. I cannot post ist here because it's about 1.2MB.

Average of ratings: -
In reply to Erich Schubert

Re: Bulk / automated course creation with phpMyAdmin

by Jason Hollowell -
Hello,

This Bulk Upload addon is really going to save my life as I have to create over 100 courses for teachers for the upcoming academic year (starts in April in Japan).

I've been experimenting with it and find that 'maxbytes' doesn't seem to be working. I used the sample csv file and got the three courses but the maxupload for each course is 8MB instead of 1KB. It should be 1KB right? The maxbytes setting in the sample cvs file is 1024.

It's not a huge deal but if I'm going to be creating that many courses I'd rather start them all off with a small upload size and let them adjust it upwards later if they have the need. So, I'd like to figure out how to make the 'maxbytes' feature work.

Thanks in advance for any help/advice.

Jason
Average of ratings: -
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by Jason Hollowell -
Ah, I missed the edit time on my previous post.

I just noticed that under Enrolments, in the course settings area, Course enrollable is set to No (the No radio button is selected) when I use the example.csv file.

What do I need to add to make this "Yes" instead of No?

I assume I need to add a column but am not sure of the heading that should be used.

Jason
Average of ratings: -
In reply to Rory Allford

Re: Bulk / automated course creation with phpMyAdmin

by Ângelo Rigo -
Hi
How can i enroll a teacher ?
which fields i have to set ?

I am using these fields but is not enrolling the teacher wich i am passing :
teacher,teacher1_account,teacher1_role,teacher1_editor,teacher1_order

Thank´s in advance
Average of ratings: -
In reply to Ângelo Rigo

Re: Bulk / automated course creation with phpMyAdmin

by Ahmad Sadeddin -
Hey all,

I am having a problem. When I upload my courses, all the users on the website get registered in them. How do I take care of this issue? Thanks!
Average of ratings: -
In reply to Ahmad Sadeddin

Re: Bulk / automated course creation with phpMyAdmin

by A. T. Wyatt -
It sound as though you have assigned a role incorrectly.

See if this helps--

"On the Moodle front page go to the site administration block -> Users -> Permissions -> User policies and see the default role for all users. If that is set as Student then you should try changing it to Authenticated user."

http://moodle.org/mod/forum/discuss.php?d=84811#p374869

If not, then you might search the forums for alternate solutions. I see this problem posted fairly often, so there must be more good advice out there!

atw
Average of ratings: -
In reply to A. T. Wyatt

Re: Bulk / automated course creation with phpMyAdmin

by Jason Hollowell -
Does anyone have this working flawlessly on 1.8.4?

I have it working sorta but have the following problems:

1. can't get it to enroll teachers no matter what I do

2. can't modify the maximum upload setting within each course. It automatically defaults to my sitewide maxbytes setting

3. can't create courses that are enrollable. They all get created with the radio button set to "no".

These 3 problems make the Bulk Course Upload tool much less useful for me. Anyone having these same problems or have these issues been resolved?

Any advice will be appreciated smile

Jason

P.S. I can get it to create courses, by the way, just not with the settings and teachers that I need.
Average of ratings: -
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by Jason Hollowell -
For #3 above I verified that there is a column called "enrollable" in the database mdl_course table. It is set to NULL when I use the Bulk Course Upload package. If I go in and change it to 1 using PHPMyAdmin then I get the result I want = Course enrolable radio button is set to "Yes".

Now, to figure out how to add this to the Bulk Course Upload package.....? I looked through the courseupload.php file but since I'm barely a beginner at PHP, I couldn't figure out how to add it in...? Tried just adding a column in the csv file but got an error saying that 'enrollable' is not a known field. Should have figured it wouldn't be that easy smile

Jason

P.S. Still also can't figure out why I can't enroll teachers or why the maxbytes feature isn't working....?
Average of ratings: -
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by Jason Hollowell -
I managed to get #2 & #3, from my post above, resolved. I added a little code (amazing considering my beginner level PHP) and now can add a column entitled 'enrollable' to my csv file with 0 or 1 (0=No, 1=Yes) and the subsequently created courses (in 1.8.4+) have the correct 'enrollable' setting. 3 would allow for an enrollment timespan or window but as I didn't need that feature, I skipped it for now.

Also figured out why my 'maxbytes' entry was having no effect when I created new courses. In the php file the maxbytes setting was locked down with an actual number rather than allowing for input from the csv file.

Still haven't figured out why I can get the enroll teachers part of the script to work but will attack that next. It's probably above my level but I'll give it a shot.

Will contact the developer of this addon and see if he is interested in adding the changes I made to the package that is maintained for the Modules and Plugins page.

Jason
Average of ratings: -
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by A. T. Wyatt -
This sounds great! Please keep us posted on how it goes!

atw
Average of ratings: -
In reply to A. T. Wyatt

Re: Bulk / automated course creation with phpMyAdmin

by Jason Hollowell -
Well I thought I'd contact the developer (Rory Allford) but it appears that he is no longer registered on the Moodle site....? Or at least not enrolled in the "Using Moodle" course.

So, here are the details of what I've done so far:

1. To allow the maxbytes column in my csv file to function properly (have the script read it), I changed line 551 in the attached file from a fixed value to ' ' so that it would read from my csv file. That seems to have solved my problem.

2. In order to be able to add a column in my csv file entitled "enrollable" I added it in the function on line 350, to the "if" clause on line 398 (twice), to the array on line 558, to the validation array on 589, to the "switch" command on line 747, and to the help table on line 836. Those additions allow the script to read the "enrollable" column from the csv file and write the value to the appropriate table and field in the database. Or at least it seems to be working that way for me.

Could someone that has more PHP knowledge than me (that shouldn't be hard smile) take a look at this and check to see if what I've done is appropriate? I've attached the uploadcourse.php file here.

Also, I've studied the enrol teachers option and can't figure it out. In my case, the script seems to be making it through "case 6" of the validateas function but when it (the script) runs I get an error saying that the teachers could not be enroled. The exact error message I get is:

Could not enrol teachers for course with ShortName t108

The short name of the course I created was t108. The next line says that the course could not be fully created due to errors but it actually does get created with no problems that I can see. The only thing is that the teacher I selected is not enrolled in the course as the teacher or in any other form.

Any help with this will be appreciated.

Jason

P.S. I'm assuming that for teacher1_account I can use the teachers name (first last) just as I would when searching for the teacher from >Site Admin>Users>Accounts>Browse list of users. I hope that assumption is correct?

Average of ratings: -
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by A. T. Wyatt -
I used 
lastname1.firstname@email.edu

as the email account as the teacher1_account value.

Seems to me that I tried some other things, but the email is the one that worked to assign the teachers. You do have the user accounts already on the system? I always had to create all the accounts and THEN use the script to create the courses and assign teachers.

atw


Average of ratings: -
In reply to A. T. Wyatt

Re: Bulk / automated course creation with phpMyAdmin

by Jason Hollowell -
ATW,

Thanks. I tried using the e-mail address that is set for the users I want to enroll as teachers. Is that what you mean here? I do have the users already in the system. So I just need to assign them to a course as the courses are created.

Unfortunately, it didn't work for me. sad I looked and copied the exact e-mail address of the teacher I was trying to assign but got the same error message as before. ??

Any ideas will be appreciated. I'm starting feel a little 'under the gun' as I need to have 200+ courses up and ready (complete with attendance records) by April 1st surprise

Jason

P.S. Sean, thanks for the update on Rory. I hope he's enjoying some good beer. smile
Average of ratings: -
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by Jason Hollowell -
Here is an updated version of the Bulk Course upload tool for 1.8 for anyone who is having trouble with the one that is available from the Modules and Plugins page.

I could not get that one to do everything it was supposed to do on 1.8.4+ It would not allow for setting the maxbytes (upload limit) because the value is hard coded into the script, would not enroll teachers on 1.8.4+, and would not allow for the 'enrollable' setting.

This updated version was provided by Jeff Church (here) and works perfectly on my instance of Moodle (1.8.4+).

The only small change I made to it was the addition of a short explanation of the terminology necessary to get the desired role for the teachers that you are assigning for each course (teacher = non-editing teacher, editingteacher = Teacher with all editing permissions).

Hope it helps someone else as much as it has helped me smile

Jason
Average of ratings: -
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by Alan Zaitchik -
Does this version work with Moodle 1.9, and must one make any edits to admin/index.php, as Rory directed for the original version of his code?
Excuse my confusion please, but I am somewhat unsure what is needed to install and run this (or the original) bulk loaded under 1.9.
Thanks,
Alan
Average of ratings: -
In reply to Alan Zaitchik

Re: Bulk / automated course creation with phpMyAdmin

by A. T. Wyatt -
Greetings, Alan!
I have used this code (in the post you replied to) on 1.9. I did not alter anything, only uploaded uploadcourse.php to the admin folder.

It did not work when we had flexpage installed; when we removed flexpage, it worked like a charm!

So in my experience, the answer is yes it should work for you!

atw
Average of ratings: -
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by Fernando Rebelo -

Hello,

I tested this code on Moodle 1.9 and works fine, except the maxbytes as i do not define the value in CSV file. I have problem to inport in linux when the category and course names have special characters. So, i altered the script to work with this.

1) MaxBytes Default Value

Line 631:
'maxbytes' => 2097152,
to
'maxbytes' => get_max_upload_file_size(),

Line 383:
global $CFG;
to
global $CFG, $optional;

Line 413: added this code
if (strstr($query, 'maxbytes') === FALSE) {
 $query = $query.'`maxbytes`,';
        $exist_maxbyes = false;
}

Line 437: added this code
if (isset($exist_maxbyes) && (!$exist_maxbyes))
        $query = $query.''.$optional['maxbytes'].', ';

2) Category Name Length


Line 219:
(strlen($item) <= 30) or csverror('Invalid value for field '.$fieldname.' (Category name "'.$item.'" length &gt; 30). '. ...
to
(strlen($item) <= 255) or csverror('Invalid value for field '.$fieldname.' (Category name "'.$item.'" length &gt; 255). '. ...

I altered the category name max lenght from 30 to 255 (the database field length)

3) Course ShortName Length

Line 654:
'shortname' => array(1,15,1),
to
'shortname' => array(1,100,1),

I altered the course shortname name max lenght from 15 to 100 (the database field length)

Line 944:
<tr><td>shortname</td><td>String(15)</td></tr>
to
<tr><td>shortname</td><td>String(100)</td></tr>

 I altered the course shortname name max lenght from 15 to 100 (the database field length)

4) To work with strings having special characters I altered the mystr function to force the UTF-8. It always works because the file is supposed to be in UTF-8 format.

 Line 314:
        return '\''.mysql_real_escape_string($value).'\'';;
 to
        return '\''.mysql_real_escape_string(mb_convert_encoding($value, "UTF-8")).'\'';;

Average of ratings: Useful (1)
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by Sean Keogh -
Rory is currently very busy studying at University in Oxford, so I suspect he hasn't got much free time at the moment for participating in the Moodle community (I guess all his free time, if he has any, will be spent at the pub, or at parties wink )

Sean K
Average of ratings: -
In reply to Sean Keogh

Re: Bulk / automated course creation with phpMyAdmin

by Greg Rodenhiser -
Has anyone come up with a way to run this php automated via cron? I've tried running it both directly from the command line and via cURL but get hung up on the authentication.

Ideally I'd like to devise a way to have this script automatically process a feed file at a set interval of time. This way I don't have to manually run course creation feeds as new courses are added.
Average of ratings: -
In reply to Greg Rodenhiser

Re: Bulk / automated course creation with phpMyAdmin

by Chad Parmentier -

I too have been working on a way to execute the upload courses file via cron. Here are my thoughts as a novice:

1. In order to run this via cron the php would have to execute the .cvs file from a folder in moodle. The php would have to have the manual upload part taken out and instead execute the .cvs file from a known location such as /moodledata/1 or something.

2. Then a .cvs file could be dumped in that folder daily and uploadcourses.php could be cron'd to run a couple times perday automating the whole process, and making this a fairly universal solution.

I am at Jackson Community College and we are willing to collaborate with other institutions/schools to complete this process.

Average of ratings: -
In reply to Greg Rodenhiser

Re: Bulk / automated course creation with phpMyAdmin

by Peter Easton -
Did you ever get any responses to your post?
I'm trying to figure it out also, could you share what you've done so far?
it would be a big help.


Thanks
Pete Easton
peaston@pratt.edu
Average of ratings: -
In reply to Jason Hollowell

Re: Bulk / automated course creation with phpMyAdmin

by Mike Worth -
I have used uploadcourse.php to create a whole load of courses with teachers enrolled, followed that with creating a whole load of metacourse links (via a script of my own). The teachers enrolments haven't gone up to the metacourrses (although when students are enrolled their's do). Do I have to unenrol and renrol all the teachers?

Thanks,
Mike
Average of ratings: -
In reply to John Molohan

Re: Bulk / automated course creation with phpMyAdmin

by John Isner -
This is one of many discussions related to the idea of bulk course creation. I'm visiting each of these discussions to try and round up support for an important tracker issue, MDL-13114 "Include bulk course create and remove with Moodle through the use of CSV files."

If you would like to see bulk course upload become a standard Moodle feature, please vote for the issue and add your comments!
Average of ratings: -
In reply to John Isner

Re: Bulk / automated course creation with phpMyAdmin

by Manuel NT -
Hi all , this topic is wat i searching for months; I want create sql statments create users , courses into categories and enrol this users into courses ;

I studing database schema form moodle but i dont have success to create a single user and create a course into a category and enrol this user ; I hope person on this topic give me a ligth ... Forgive poor english , Thanks for all .
Average of ratings: -
In reply to Manuel NT

Re: Bulk / automated course creation with phpMyAdmin

by Manuel NT -
I forgot to say , i use moodle 1.9.2 ; My system has courses and users in others databases but the structure is to much complex to use external autentication and external enrol with DB , so I wrote some scripts and sql statments to catch some extrernal information to a CSV file , so I want to create a similar script to inser a user in moodle , inser a course (check if exists) and enrol a user to that course ; But I don,t understand cleanly how moodles work on database schema , the script on this topic is very useful , , but if someone can explain wath fields are modifieds when i create a new user or a new course in a existent category , and how enrol this user only with SQL be easyst to finish it .. . thanks for helping. Obs: I,m a PHP programer and have a lot of experiency with mysql to ;


Average of ratings: -