uploaduser.php enhanced to upload also courses

uploaduser.php enhanced to upload also courses

autor Rosario Carcò -
Počet odpovědí: 41
Dear all,

I took the existing code of /admin/uploaduser.php and uploaduser_form.php and enahnced it to create also new courses.

The code is called when you click on the link in Admin->Users->Accounts->Upload users. After Installation of my code you will find a new entry there with a link to uploadusersandcourses.php

  • In the first pass the uploaded file is parsed for new courses to upload/create.
  • In the second pass, the original code gets executed as it was to create new users and/or enrol existing ones.
Could you please test it on your test-servers and let me know what you think of it?

Could an experienced moodle programmer have a look at it also to judge whether this code could be used to replace the existing uploaduser.php?

You will find a readme file in the zip file with installation instructions.

I included also a revised help file and needed language files.

In debug modus I found a minor php-notice if course1, course2, etc. are present in the first line of the csv file, but without valid data. So I corrected this issue.

NOTE: I have planned also to engineer this code down to make script out of it which could be run as an unattended cron-job, i.e. without going through the mentioned Admin->GUI

Rosario

Průměr hodnocení: Useful (2)
V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -
There is a second version with courseTemplates working fine, i.e. you can put a path to an existing backup-file you want to use as template to restore into the newly created course. Example of path: 20/courseTemplate.zip, i.e. the directory where it is stored. See the README file for more informations.

Again, feel free to test and report back.

Rosario
V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Martin Vögeli -
Will this (or something similar) be in the core one day? Would be great! Best wishes, Martin úsměv PS: 5 years Moodle @ ZHAW anniversary!
V odpovědi na Martin Vögeli

Re: uploaduser.php enhanced to upload also courses

autor Tim Hunt -
Obrázek: Core developers Obrázek: Documentation writers Obrázek: Particularly helpful Moodlers Obrázek: Peer reviewers Obrázek: Plugin developers
No reason why it shouldn't be, if it is generally useful and good enough.

The procedure is Development:Overview#Minor_Development. (That is, get a nicely polished patch attached to a tracer issue.)
V odpovědi na Tim Hunt

Re: uploaduser.php enhanced to upload also courses

autor Mark Stevens -
Would MDL-13114 qualify? This is really needed, given all of the forum discussions, and code written to try to get this going (CONTRIB-605).
V odpovědi na Mark Stevens

Re: uploaduser.php enhanced to upload also courses

autor Tim Hunt -
Obrázek: Core developers Obrázek: Documentation writers Obrázek: Particularly helpful Moodlers Obrázek: Peer reviewers Obrázek: Plugin developers
Yes. Those would be the ones.

And with that many votes, someone will deal with it eventually. However, lots of other things in things on the Moodle 2.0 roadmap, so don't hold your breath.
V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -
Here is version 03 with the promised code you can run as an unattended batch/cron job.

Please read the readme file and my comments in the code regarding execution security and privileges.

To debug and test the script you are able to call it by web, e.g.

https://yourMoodle.com/admin/uploadusersandcourses_silently.php

or by wget as a cron job, e.g.

wget -q -O /var/log/uploadusersandcourses_silently.log https://yourMoodle.com/admin/uploadusersandcourses_silently.php

or even by the php interpreter on a command line. I could not test the latter, as on my server php execution on the command line is disabled. But it should work, as I copied the necessary code from admin/cron.php which is designed to do so.

Regarding the mentioned Threads/Contribs I have contacted Anthony Borrow to see what could be my humble contribution. I fear that there is so much code already I didn't even know of and - even worse - I did not find, when I looked for it. If I would have found it, probably I would not have started to code my own solution.

Any feedback is greately appreciated, Rosario
V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -
Using a backup of a course as template works fine. There is only one strange thing: the Enrolment Key is not restored from the backup, i.e. the setting is empty after the restore.

Does anybody know why?

Depending on the aswer I will have to code this option myself or check and modify the script import_backup_file_silently to restore also the Enrolment Key.

Rosario
V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Anthony Borrow -
Obrázek: Core developers Obrázek: Plugin developers Obrázek: Testers
Rosario - Is this code related to a particular tracker issue? If so, would you mind uploading it to the appropriate issue. Peace - Anthony
V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Michel Sandri -
Hi. I don't know with u are developing this tools. I'm testing it to use on my institution and i have a dout about the category creation. Your tool don't create the category, just use one still created right?! have any update?!
Thanks.
V odpovědi na Michel Sandri

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -
Yes, my code does create ONLY COURSES in EXISTING categories. As I mentioned in one of my posts, there are many many bulk operation scripts around in the contrib moodules & plugins section.

I could also introduce category creation but I have no time for the moment.

I was also asked to split up the code so that we could:

- upload categories

- upload users

- upload courses

individually and integrate the code into the accourding Admin-Block -> users and Admin-Block -> courses menus.

But you are right, as my script is also intedend for silent upload as cron/batch script, it should also be able to create categories if they do not exist yet.

As I am working on two Blocks (myCourses and siteNavigation), I will resume this around Christmas I hope.

Rosario
V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -
Here is version 04. I have only a few minor edits/enhancements or bug fixes.

Rosario
V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

I will start working on Version 05 in the next days. Here are the features requested by our Moodle-Admins:

- option to rename a course

- option to copy a course

- option to archive a course

- option to delete a course

So I will add another column in the Excel/CSV File: COURSEACTION, which can be:

- new (if empty or new, a new course will be created, as before, using a given path to an existing backup-zip-file as course-template)

- rename (based on course-idnumber, which should be unique) to given short and fullname

- copy (create a new course using the own backup-zip-file as course-template. I am not sure on how to achieve this, but I could use course-idnumber again to find the path to the own backup-zip-file. The new course would then receive the given short and fullname and course-idnumber should always be identical to shortname). Pay attention, this could be a very bad idea because duplicating whole courses in unattended mode could fill your data-disk.

- archive (not clear, has to be defined yet. But this should maintain the course's actual link while creating the copy as a new course or simply copying the backup-zip-file of the course into a course or category defined to receive all archived courses.)

I will report back during my work and when I am finished, I could add also the requested option to create categories if they do not exist yet.

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Marin Sok -

Hello Everyone,

I want to create an import courses in PHP script for  Moodle 2.0

Does everyone has sample code?

I couldn't figure out to use create_course function.

I'm new to PHP.

Any sample code of call create_course function, will be appreciate.

 

Thanks,

 

 

V odpovědi na Marin Sok

Re: uploaduser.php enhanced to upload also courses

autor Johnny America -

Yes, anyone has an example of how to use create_course function?

V odpovědi na Johnny America

Re: uploaduser.php enhanced to upload also courses

autor Mark Johnson -
Obrázek: Core developers Obrázek: Particularly helpful Moodlers Obrázek: Peer reviewers Obrázek: Plugin developers

There's a tool for uploading courses using a CSV flatfile which I recently updated for 2.x: http://moodle.org/mod/forum/discuss.php?d=173705

V odpovědi na Mark Johnson

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

Thanks for this it will help me to convert my own uploadusersandcourses.php script to Moodle 2. I feel it very strange that such an important function was not incorporated into core. Even Tim Hunt speaks for it in his post here.

P.S. for "new to php", then you have no chance unless you invest a lot of time. It is better to use scripts others have developed to a ready to use level. My own script is not at this stage yet. Depending on your needs you have to tune the code yourself, which can only be done, if you understand what my code does. I am used to comment every line to help you reed my code. So it is up to you and I can only answer the questions you may have.

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

Please follow progress also in this tracker issue:

http://tracker.moodle.org/browse/MDL-13114

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor ursula macfarlane -

Hi,

I'm gatecrashing here - sorry - but as our organisations IT trainer I really want an automatic process that will upload new users without having to go into Moodle myself and upload the .csv file.

Our network login process creates the users in email and on the network. This is added into a spreadsheet. We would like to run an overnight process that picks up this spreadsheet and creates the Moddle accounts.

IT guys I am talking to here prefers that I get into the discussion forums to find the answer! So I'm playing middleman......

Another organisation  is hosting our part of the moodle server so I guess we can't actually do anything, but they do have an LDAP over a VPN.

For  our technical infrastructure Andy says this;

"Can we configure a persistent VPN to you guys, then call an LDAP script from our clinical account creation script, that could create the Moodle account in your LDAP server?"

I think they are stumped at the other end......

cheers,

Ursula.

V odpovědi na ursula macfarlane

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

LDAP can be scripted, indeed. But LDAP scripts can only be executed on LDAP Servers. So if you use LDAP authentication like I did for years, before switching to Shibboleth authentication, you would set up the new users on your LDAP Server first. Then, if I am not mistaken, the Moodle LDAP-Module does not offer any "overnight" synchronisation of the Moodle users with your LDAP Database, but you could export your LDAP Database to something you can process with Moodle's EXTERNAL DATABASE Authentication Plugin. See users->authentication->external database where you can choose a lot of different "databases" even plain csv files.

But pay attention to what "synchronisation" means in this auth-plugin, so as not to loose Moodle users accidentally deleted in your LDAP database, unless this is what you want to do.

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Iñaki Arenaza -
Obrázek: Core developers Obrázek: Documentation writers Obrázek: Peer reviewers Obrázek: Plugin developers

LDAP can be scripted, indeed. But LDAP scripts can only be executed on LDAP Servers.

As long as you can connect to the LDAP server (more precisely, to the LDAP service ports), you can run the scripts from anywhere you want úsměv (I've done it many times in the past )

Then, if I am not mistaken, the Moodle LDAP-Module does not offer any "overnight" synchronisation of the Moodle users with your LDAP Database

It sure does úsměv It's under auth/ldap/auth_ldap_sync_users.php in Moodle 1.x and auth/ldap/cli/sync.php in Moodle 2.x

Saludos. Iñaki.

V odpovědi na Iñaki Arenaza

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

Iñaki, thanks for the details. You are rigth (as ever). I had some confused remembrance of the auth_ldap_synch_users.php but did not find it in the GUI auth-ldap-settings. But you are right, it exists nevertheless, and I am remembering now that I did study that code before coding my uploadusersandcourses.php - my brains are getting older...

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

Dear all, I am working on a version for Moodle 2.3 now. If the Moodle API calls didn't change too much, I should be able to post the first beta in the next 7 days. The only headache will be to use the new Backup-Zip-files to copy/duplicate an existing course as the file names and directories are no more in plain text inside the course directory. May be I will need your help to program this.

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

Just finished the first mockup. I will upload my first beta here soon. Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

Dear all, here is the first beta. To install proceed as follows and described also in the readme file:

  • unzip everything onto a local drive
  • copy the whole uploadusersandcourses folder/directory to you moodleDir/local/ directory
  • in the Moodle GUI visit the NOTIFICATIONS link to load that new local plugin

You will then find automatically a new Menu-Option Upload users and courses in your siteAdministration->Users->Accounts Menu.

For the moment, to make our first tests, only the interactive code is there. Next week, if your echo is positive, I will copy the new code into the uploadusersandcourses_silently.php script, test again and release it here.

The version numbers have changed a little bit: the first two digits are meaning the Moodle version my code was tested on and works fine on. So 23.01 has been tested on Moodle 2.3

Rosario

 

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

And here is the complete BETA, with _silently.php version.

a) use it on a TEST-SERVER until we are through with testing

b) REMEMBER the code can now be neatly integrated into Moodle as external plugin but this does not mean you can simply install and use it!

NO, No, no...

Before using it, you must have a look at my readme file. You must have a look at the code because it reflects mainly the needs of my University.

BUT with little effort you can modify the code to reflect your own workflow and adapt it for your needs. There is no magic in it. And if you need my help to tailor the code to fit your own needs, just let me know.


In the silently-version I left things so that you can use/run it by a web-call, so as to see immediate output in your browser. To make it run in cli on the command shell, remove // from the line where

define('CLI_SCRIPT', true);

is defined. Toggle also the flag $checkadminuser to FALSE, as when runnig from command line you suppose that no other user is able to run it.

Calling by web means https://yourMoodle.com/local/uploadusersandcourses/uploadusersandcourses_silently.php

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Nigel Cunningham -

Thanks for this Rosario.

I'll certainly be looking at it and giving it a try because I've gone from manually creating users and enrolments to a process of using a CSV export from our Student Database, combined with running three scripts and two visits to the Upload users page. If I can use your solution, I might be able to make the whole process fully automatic, which would be great!

V odpovědi na Nigel Cunningham

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

You are welcome. As I said the 1.9 version was in production since december 2011 in silent, full automated version. And I used it in the GUI-Version ever since I developed and published it here. Now the 2.3.1 Version is in production since september 14th, even if I say it is a BETA version here...

I will enhance it and clean up the code a little bit, but I want the code to be clear and understandable also for newbies, so that everyone can tailor to fit heir own needs. I think I will add some admin-config-options to facilitate using the GUI-version without having to set the variables in the code itself, to make it more ready-and-out-of-the-box.

But those who want to automate, must have a look into the code to avoid surprises.

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Mark Jones -

Rosario,

Where do i get a copy of the production version? also it only works with Moodle 2.3.1 and above...correct? mark

V odpovědi na Mark Jones

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

Mark, for the moment the above BETA Version is what is in production for us... it works fine for us.

I will upload the next version as soon as I have time to make little changes to make the GUI-version working out-of-the-box.

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor Daniel Phillips -

Hi Rosaio,

Your mod looks promising.

I am wondering if I could use it to semi-automate backing up and restoring of courses.

At my place we use an integration to create the courses from our SMS, but then have to manually backup/restore or import the course content.  The usual process is to restore content (or import) from the most recent iteration of the same course that has run previously e.g. We will restore the Semester One course for the Semester Two course.

Can this mod achieve this by adding the appropriate info into the spreadsheet?  Would be great if it could - save me a ton of time úsměv

I appreciate your help.

Cheers
Dan

V odpovědi na Daniel Phillips

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

YES, you will find all the code you need to do so.

The actual logic is this way:

  • read in csv file
  • in first pass create courses that do not exist already
  • if a course-template file is given in the file or a course to copy from, the same procedure is started for "new" and "copy" courseaction. The existing course is backed up on the fly and restored to get a copy of it which becomes the new course
  • in the second pass users in the csv file are enrolled in those courses or in any other existing course
  • to create the courses in certain categories you can either use a category-translation-table or deactivate the table and use course-category-IDs directly in the csv file

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor R.T. Brown -

Hi Rosario,

My apologies if the answer to my question has been posted elsewhere. Does the script only enroll students whose accounts are created at the time of upload, or does it accommodate enrollment of students who already exist in the system?

Many thanks --

rtb

V odpovědi na R.T. Brown

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

I am very sorry for the delay. As I wrote at the beginning, the uploaduser-part does exectly what it ever did, as it is simply a copy of this code. So it enrolls BOTH users already existing or newly created on the fly. And it does not matter whether you use uploadusersandcourses to either create only courses and/or create new users and/or enrol them in existing or newly created courses.

V odpovědi na Daniel Phillips

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

Daniel, I am not sure whether I caught your idea/workflow:

>>The usual process is to restore content (or import) from the most recent iteration of the same course that has run previously

If your courses exist already, it would be enough to DEACTIVATE/HIDE and ACTIVATE/SHOW them to the Students. I think my code to read in your csv file with the simple course-short-names and one api-call to set the visible-flag to true/false would do the trick.

Another possibility would be to re-set the start-dates and/or enrolment-dates and/or enrolment-duration of the existing courses, so that access for the students would be automatically controlled by the dates you set. And with my script you would simply re-set those dates for the next term.

Finally, if you want to unenroll the previous students automatically and enroll the new ones still using my script, the best way to go is really using a backup/restore of the existing course without user data. But if you copy existing courses to new ones, of course, one day you will have to delete the old/original courses. So you have two new options here: either I finish implementing the deletion of existing courses in my script, as I had already scheduled to do, or I/you modify the script to fit your needs to unenroll previous students and enroll new ones. Even the latter should not be too difficult to implement.

Rosario

V odpovědi na Rosario Carcò

Re: uploaduser.php enhanced to upload also courses

autor marco cassisa -

Hi all!

I enjoyed Rosario Carcò's script and tried to enhance it a little more.

Yes I know, there is Piers Harding's work that allows to upload users in bulk, but it seems that uploadusersandcourses fits almost exactly to the needs of my school, at least to an extent that I couldn't resist temptation to take a little further its functionalities.

I rewrote a part of the script so that it takes a category-tree and recreates it if not exists (with this syntax:   MAINCATEGORY@@SubCategory@@SubSubCategory and without limits to the number of subcategories (except moodle'ones of course).

The category-tree delimiter can be easily changed in the script. The code is still quite dirty and can be surely optimize (idcategory isn't necessary anymore, but it is set internally, which can be avoided now), but, hey, WorksForMe! 

I sent the script to Rosario asking for a review, but he has a lot to do, as it seems.

The power of open source and free software is that everyone can contribute. My contribution is attached (just uploadusersandcourses.php file I modified - not the entire plugin! - and a sample-csv): enjoy and improve it further!

marco

V odpovědi na marco cassisa

Re: uploaduser.php enhanced to upload also courses

autor Rosario Carcò -

Yes, I was busy, but the script for Moodle 2.3 has been working and continues to work fine on our production server since one year now.

In the meantime big efforts have been made to integrate this code into core. Have a look here to follow development:

https://tracker.moodle.org/browse/MDL-13114

I still have no time to look at that code either. But I feel that my code can continue to help those who want to tailor it to fit exactly their own needs and special workflows.

As I explained Marco, the category structure at our University of Applied Sciences is something that does not change a lot. So our need is only to create courses in those given categories. If we have changes in our category-structure I simple rename the categories or create new ones in the GUI and then I have to modify my course-category-table in the code to match the new names or new categories.

But as Marco says, you are all free to use the code as is, or modify it to fit your exact neeeds.

Rosario

V odpovědi na marco cassisa

Re: uploaduser.php enhanced to upload also courses

autor marco cassisa -


To use rosario's script in moodle 2.7 it is enough to add 

$systemcontext = context_system::instance();


to settings.php @line 27 (before any other interpreted code)


Bye all, thanks rosario!

V odpovědi na marco cassisa

Re: uploaduser.php enhanced to upload also courses

autor sas ss -

Hello, thanks for the amazing work! is there any solution for 2.9? I tried $systemcontext = context_system::instance(); it removed a syntax error but the page keeps giving me blank

V odpovědi na sas ss

Re: uploaduser.php enhanced to upload also courses

autor sas ss -

Hello, I made the upload user work for 2.9, but still the create course is not it is giving me Courseaction missing or unknown, I will attach the zipped code to where I reached, I do not know if its still a deprecated code or is it something to do with $filepath = '/mnt/evento/'
 
$statusfile = $filepath . 'moodleCoursesNewStatus.csv' maybe is it comparing to it; any suggestions?