Customised Scripts

Customised Scripts

by Shane Elliott -
Number of replies: 21
Picture of Core developers Picture of Plugin developers
I probably should have announced this some time ago but .... well can't think of a good excuse.

I've added a hack to 1.6 which will allow some customisation without changing the original files.

A quick overview:

1. Switch on $CFG->customscripts in config.php (there is some more info in config-dist.php);

2. Any scripts that are called directly via a url (eg index.php, user/view.php) can now be customised by placing a file with the same name (including directories) in my_moodle_data_dir/customscripts
For example:
my_moodle_data_dir/customscripts/index.php; my_moodle_data_dir/customscripts/user/view.php

How it works (eg for index.php):

index.php includes config.php which in turn includes setup.php. At the end of setup.php is some code which will search for the respective custom script and include that if it exists (and $CFG->customscripts is switched on). There are two things to remember:

1. your custom script should not include config.php again;

2. control will return to the original script unless you explicitly 'exit' or 'die' at the end of the custom script. This means you can either add functionality to a script or completely replace it.


Hope that helps some people out there with developing, customising, whatever.

Average of ratings: Useful (1)
In reply to Shane Elliott

Re: Customised Scripts

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Triple Cool! cool

I can imagine some interesting hooks to implement following that schema (firing a lot of stuff when any script is invoked!). Just two thoughts I've had just now:

1.- As such custom scripts are executed BEFORE the real one, they shouldn't send any output to the page or the page will break (headers....).
2.- Would be difficult to add the feature to have both pre-script and post-script scripts (perhaps launching them in the footer.php or something similar. A lot of stuff that is currently calculated in the real script could be perhaps used by the post-script one?

Just quick thoughts, not deeply matured! Great job, not excuse needed!

Ciao smile

P.S.: I love hooks, hooks and hooks and these are, really a type of them!
In reply to Eloy Lafuente (stronk7)

Re: Customised Scripts

by Shane Elliott -
Picture of Core developers Picture of Plugin developers
1. yes (unless you are completely replacing the original script);
2. you can do it by completely replacing the original script and adding your own code to the end - not really a hook but functional

I'm sure the code can be improved but at least it's a start.

Check out:
http://test.moodle.com/
http://test.moodle.com/?egg=1234
http://test.moodle.com/?egg=4321 (changes info sent to the footer)

(Note to people reading this thread at some later date - the above links may not do anything as the test site is constantly being changed)

In reply to Shane Elliott

Re: Customised Scripts

by Martín Langhoff -

Shane,

couple of questions:

  • how does this work internally? (I'm thinking performance impact, security).
  • it kind-of encouranges people to not use CVS's own support for merging. Which means that they have a much harder time to remerge their customizations.

Perhaps I should explain the 2nd bit. If someone gets an anon CVS checkout, and makes changes, and goes through life doing cvs update, cvs does the right thing, keeping track of the last version from the repo checked out, the diff, and doing 'diff3' to merge the same thing up to the newer one.

Of course, every once in a while there's a conflict, but that is good. When CVS has a conflict, it means that you must pay attention, and decide whether upstream changes mean that you have to change your customization. Also, when there's a conflict or a merge, your origial file is always preserved as .#somesuch.php.v2.1.2

Customizations done in a copy of the file don't have this benefit, and are sheer hell to keep updating (once you've done an update, what to you diff it against?). So they won't update, and go through life with old, buggy versions of those files, customized.

I rather encourage people to use an anon CVS checkout, and to pay attention when the files they've customized get a merge or get a conflict. That's how we track 'lightweight' customizations (we use cogito for the heavyweight ones).

Or perhaps I'm misunderstanding...?

(Note: I hate CVS with gusto, but it does this part of the job well enough and is what Moodle uses, so it is the best soluition for this kind of scenario...)

In reply to Martín Langhoff

Re: Customised Scripts

by Martín Langhoff -
I re-read your post and perhaps I shoud clarify: the part that worries me WRT not using CVS is the "replace the whole script this way" scenario.

The "pre-run" hook part of it is a mighty good thing, and I'm with Eloy on that one ;)

So... I only _overreacted_ about that. Don't send the people in white just yet...
In reply to Martín Langhoff

Re: Customised Scripts

by Shane Elliott -
Picture of Core developers Picture of Plugin developers
All good points. And I agree with you, it's not the ideal way to customise scripts. smile

Unfortunately if the hook is there then the possibility of a complete replace is there too.
In reply to Martín Langhoff

Re: Customised Scripts

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
No-one's suggesting not to use CVS, we always had that and I'm sure we'll keep doing that. This is just another possibility that could prove useful in some cases (and has practically zero overhead). It's off by default, too.

Maybe it could use another config variable to specify the customscripts root directory.
In reply to Martín Langhoff

Re: Customised Scripts

by Martín Langhoff -
After a second re-read I see that you are talking about moodledata. That is a bad idea. Perhaps a different directory, one not writable by www-data/nobody?

To execute PHP files generally uploadable/modifiable by the user Apache runs as is a big no-no. Skodak will rip us to pieces wink (even before the hackers have a chance).
In reply to Martín Langhoff

Re: Customised Scripts

by Gustav W Delius -
I must say I share Martin's worries that people will mis-use this custom script functionality as an alternative to using a properly documented API and then have difficulties keeping their modified Moodle in synch with the main Moodle.

I just noticed that the Moodle for mobiles project is doing just that. It is selectively replacing some Moodle scripts with its own version. This will make the Moodle for mobiles project very unreliable because one never knows whether some core Moodle code has been changed in a way incompatible with the Moodle for mobiles scripts.

So to repeat what Martin already explained: modifications to Moodle should either be done right in the Moodle script files so that they can be merged using CVS or they should use an agreed API.
In reply to Gustav W Delius

Re: Customised Scripts

by Gordon Bateson -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers
I agree very much with what you say Gustav, and I hope that in the long term the modifications that are being pioneered for the Moodle for Mobiles project will make it into the main Moodle script files.

However, I think there are several practical reasons why in the short term, it makes sense for Moodle for Mobiles to work with "customscripts".

First, the human and financial resources to complete the required programming in one go are not available, but it is possible for the project to make progress using "customscripts". Getting Moodle working on mobiles requires changing every script in that produces output (just about all of them!), because Moodle assumes it is outputing content for a PC browser on a big, wide screen. By using "customscripts", the project can progress step by step and stay within budget while producing a working end result.

Second, I don't think anyone in the Moodle for Mobiles team has write access to the main Moodle scripts, so changes to core scripts would have to be requested via the moodle-bugs site. I know from personal experience that it is several hours work to craft a moodle-bug report that it likely to be noticed and then track it, whereas it takes a few minutes to implement the change myself.

A third advantage of using "customscripts" at this point is that it allows a working model which can later be used to show the maintainers of Moodle's core scripts and modules what modificiations would be necessary to get Moodle up and running on Mobiles. Asking maintainers to "make your module work for mobiles" is a tall order, whereas asking them to implement code changes that have been well tested has a much greater chance of success.

For these reasons then, it may be a good idea to use "customscripts" in the short term, with a view to making the necessary changes to Moodle scripts as soon as possible. In fact this has been what has happened for "Cookieless sessions" and the "customscripts" functionality.

By the way, I am neither the lead developer nor a sponsor for Moodle for Mobiles, so these views are purely my own and may or may not reflect the actual reasoning behind the decision to use "customscripts" smile

best regards
Gordon
In reply to Gordon Bateson

Re: Customised Scripts

by Gustav W Delius -
Gordon, thanks for your explanations. I must admit I haven't been following the Moodle for mobiles project. I am glad to hear that the use of customscripts is not seen as a long-term solution but as a useful development step to demonstrate what kind of changes are necessary to core Moodle to allow this kind of functionality.
In reply to Gustav W Delius

Re: Customised Scripts -Moodle for Mobiles projects use of

by Jamie Pratt -
Hi Gustav and Gordon,

Thanks for chipping in in defence of my design decisions Gordon!

I hope that Martin Langhoff and Martin Dougiamas read this thread as well as they seemed doubtful of my design decision and I do think the decision was the best for this project. It is possible that I've missed something in my planning and I'd be very interested if someone could point this out to me.

Gustav I can understand your concern about the Moodle for Mobiles code. As Gordon points out though html and javascript output is woven into just about every part of Moodle. And so to get Moodle code to work on the majority of Japanes mobile phones which really have VERY limited capabilities is very tough. Japanese mobile phones :
  • Don't support javascript, cookies, the redirect method used by Moodle, css, tables or many other html tags.
  • Older phones choke on a page including images which is more than 2Kbytes!
  • The screen is smaller and so we really need the ability to completely rework the user interface.

If Moodle completed seperated it's html output from the program logic using templates still a lot of modification would need to be done to the Moodle code to get it working on a Japanese phones because of the constraints of the small screen, the use of javascript and other factors.

We did get cookieless session support included in the main Moodle distribution thanks to Martin Dougiamas's interest in this. Still the code needed to output pages for a Japanese mobile phone involved significantly reworking the Moodle code. About the only thing we are doing exactly the same with the Moodle for Mobiles code is storing stuff in the db in the same way, in the same db tables.

Customscripts provided the best way to do this. We used completely reworked scripts to generate pages for Japanese mobile phones, basically I have stripped a lot of stuff from the Moodle code and in many cases had to replace code that relied on javascript or or other functionality not available on a mobile phone with something else.

Other Options

There were two other options that I saw :
  • to make massive changes to the Moodle core code put a lot of if switches and so on in the main Moodle code.
  • or to filter the output of Moodle to control as much as possible the output of Moodle by making changes in CFG settings and then stripping out unwanted tags and manipulating the xhtml output to output it in a form suitable for mobile phones.
The first way would also be very difficult to maintain. The second option would not give us enough control over the content and the way we want to display it on a mobile phone. The second option could also have been potentially very slow depending on how much manipulation we were going to try to do. I considered it important that on a mobile phone with very small page size that pages should be output as quickly as possible.

My plans to track changes in Moodle code

The structure of the code for the Moodle for Mobiles project is very similar to the structure of the code of Moodle itself. What I plan to do is to track the Moodle code in CVS and release a version of the code that will work with the stable version of Moodle 1.6 when it is released. Barring any major changes in Moodle 1.6 stable after it is released then the code should work well besides 1.6 for 1.6s life time.

If there are changes that break the Moodle for Mobile code then I will update the Moodle for Mobile code. There will need to be updates of the code for at least for every major release of Moodle. I think it makes sense that I track the changes in Moodle and merge them into the Moodle for Mobiles code or make the appropriate changes in the Moodle for Mobiles code and that this will not be that difficult to do.
In reply to Jamie Pratt

Re: Customised Scripts - danger in casual use

by Gustav W Delius -
Jamie, I am very much regretting that my post mentioned the Moodle for mobiles project as an example. Clearly you have very thoroughly thought about what you are doing. I was worried about people using the custom scripts capability without realising the difficulties that will arise when Moodle is upgraded. So Moodle for mobiles was a very bad example for me to choose because there was indeed no other way of getting your project on the road.
In reply to Gustav W Delius

Re: Customised Scripts - danger in casual use

by Jamie Pratt -
No problem at all Gustav.

Martin D asked me about why I used customscripts during the developers conference, I believe you were there? Since I didn't do a very good job then of explaining my design decision, off the cuff, I've been wanting to spend some time explaining a little why I chose to do things this way. So this discussion has given me the chance to do that.
In reply to Shane Elliott

Re: Customised Scripts

by Urs Hunkler -
Picture of Core developers

Shane, purely great smile

From the theme development point of view this approach bears the possibility to develop changes in the page structure for better styling possibilities without changing the source code. I for example can then test these changes on live sites. Testing themes on faked test sites is always frustrating, because the real world user pages will often break what looked so cool on the test site.

Martin proposed another variable for the customscripts root directory. Great - could I then save the changed pages into the theme folder and use it with the theme? This would make the working and development process even more comfortable.

Special page layouts needed to integrate Moodle into Corporate Intranets for example would be possible without hacking Moodle and without update hassle.

Urs

In reply to Shane Elliott

Re: Customised Scripts

by Jamie Pratt -
I put this idea into the bug tracker http://moodle.org/bugs/bug.php?op=show&bugid=4501&pos=0

Hi,

I really like the custom scripts code.

Thought it might be a nice idea to make a one line change to the $CFG->customscript code. I'll be using this for the Moodle for Mobiles project and I think that we could solve a couple of issues by changing :

    // Strip wwwroot out
    $scriptpath = str_replace($CFG->wwwroot, $CFG->dataroot.'/customscripts', $urlpath);

to

    // Strip wwwroot out
    $scriptpath = str_replace($CFG->wwwroot, $CFG->wwwroot.$CFG->customscripts, $urlpath);

We then would leave $CFG->customscripts unset to disable customscripts or set it to false; or set it to a path below wwwroot if we want to use customscripts.

and then change the config-dist.php from :

// Custom scripts should not include config.php
// Warning: Replacing standard moodle scripts may pose security risks and/or may not
// be compatible with upgrades. Use this option only if you are aware of the risks
// involved.
//      $CFG->customscripts = true;

to :

// Custom scripts should not include config.php
// Warning: Replacing standard moodle scripts may pose security risks and/or may not
// be compatible with upgrades. Use this option only if you are aware of the risks
// involved.
// Pathname should be relative to www root and start with a
// back slash :
//      $CFG->customscripts = '/pathtoyourcustomscript';
//
// leave $CFG->customscripts empty or set it to false to
// disable customscripts.

Jamie
In reply to Jamie Pratt

Re: Customised Scripts

by Shane Elliott -
Picture of Core developers Picture of Plugin developers
Finally committed changes to CVS.

$CFG->customscripts is now a full directory path to your custom scripts eg
$CFG->customscripts = '/home/example/customscripts';

This now gives you the freedom to place them wherever you like, ideally in a place that is neither in the "web space" nor somewhere that is writable by the web server.

Some other tidy ups done as well.

[Edit] Bug 4501 now closed
In reply to Shane Elliott

Re: Customised Scripts

by Gordon Bateson -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers
Hi Shane,
thanks for implementing the "customscripts". It's very cool and very crucial for the Moodle for Mobiles project approve

I was having some problems with the "custom_script_path" function.
  • the function returns a directory path, not a script path, if it is called with an empty $urlpath
  • the function removes "C:" from the start of $CFG->dirroot on Windows servers
  • the function does not correctly detect when to append "index.php"
I have reported these issues as bug 4827 on the bugtracker and suggested some soultions. Please could you have a look and tell me if it would be possible to apply my suggested solutions smile

many thanks
Gordon
In reply to Gordon Bateson

Re: Customised Scripts

by Shane Elliott -
Picture of Core developers Picture of Plugin developers
Hi Gordon,

Just read your post. Thanks for all that. I must admit I never tested it on a Windows system.

I'll go over your code in the bug tracker and integrate with the changes I've just made.

Cheers!
In reply to Shane Elliott

Re: Customised Scripts

by Gordon Bateson -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers
Hi Shane,

if you would like access to a Windows server for testing the "custom_script_path" function, just let me know and I'll arrange for you to be able to hack into mine smile

all the best
Gordon
In reply to Gordon Bateson

Re: Customised Scripts

by Shane Elliott -
Picture of Core developers Picture of Plugin developers
Following a request from Urs at the developers conference earlier I'm posting an update on the custom scripts here.

I've incorporated the code changes by Gordon to make it work on Windows systems - see Bug 4827 - many thanks to Gordan for his work.

As stated previously, the $CFG->customscripts variable which is set in config.php should now contain a path to the custom scripts directory rather than be a simple boolean. The reason for this is that ideally you would want the directory to be outside of the web space AND not in the $CFG->dataroot directory either (as this is writeable by the web server).

I'm thinking it may be useful to add a warning for anyone upgrading from 1.5 to 1.6 about changing $CFG->customscripts if they have not already done so. Presently it will just silently fail if it does not detect a valid path name.

Cheers,
Shane.
In reply to Shane Elliott

Re: Customised Scripts

by Urs Hunkler -
Picture of Core developers

Thanks for your fast posting Shane.

The replacment of the index.php page to integrate special functionality worked perfectly.

Urs