I sent mail to Martin probably 2 or 3 months ago. He is, understandably, swamped, but at that time that's what a document somewhere on this site said to do. I'm a sysadmin and PHP coder, and I'm maintaining Moodle for a university installation, and I wrote the uploadmulti assignment type, but have held back on implementing features that would force me to code outside of the mod/assignment/type/uploadmulti directory because then when I upgrade moodle things will get blown away.
You can find the module here --> http://www.linuxlaboratory.org/index.php?title=Category:Downloads
Other users have found this module useful, and in the course of maintaining this installation, it's been requested that I make some small changes, fixes or alterations that actually make sense for moodle at large. I did submit one or two feature requests, but they've apparently either fallen on the floor, or been deemed not useful or something.
So how can I help to maintain and develop moodle without completely forking the code for my site?
brian.
Brian -
When you update code using CVS, the CVS utility will check your current files and the repository's files, and try and merge changes from both of them. Therefore, it is actually possible to make changes, even to core Moodle files, and then when you run cvs update -dP it should not "blow away" the changes you've added.
There are times when the CVS utility can't successfully carry out the merge, in which case it will tell you and you would have to do some manual merging. But I've never had that happen to me (neither with Moodle nor with other projects).
It can be frustrating trying to join in the periphery of the coding for projects like this - I'm in a similar position as you re certain small changes to the lib files, etc!
Dan
When you update code using CVS, the CVS utility will check your current files and the repository's files, and try and merge changes from both of them. Therefore, it is actually possible to make changes, even to core Moodle files, and then when you run cvs update -dP it should not "blow away" the changes you've added.
There are times when the CVS utility can't successfully carry out the merge, in which case it will tell you and you would have to do some manual merging. But I've never had that happen to me (neither with Moodle nor with other projects).
It can be frustrating trying to join in the periphery of the coding for projects like this - I'm in a similar position as you re certain small changes to the lib files, etc!
Dan
You could also ask Martin for CVS access to /contrib/ (which is easy to get) and maintain your module there for a little while. This way it will be exposed to more people, who will hopefully start using it and giving you feedback. Once it's "in the open" it could reach critical mass and get into the main distribution, achieving your goal.
I'm just beginning to work with Moodle, but my situation is much the same. Working on an installation for a small college. I need to make a few small changes to get Moodle ready for next semester. But my responsibilities are much larger than just maintaining a Moodle installation. For Moodle to be viable, I need to push my changes upstream. (And I'd be happy to rewrite them until they are in an acceptable form.)
I had hoped that I could hang out and continue submitting patches. Just getting patches accepted would be fine. But getting CVS access would make my life easier. But your story is not a story of hope.
If you are like me maintaing your own module in contrib, is not an acceptable solution. Forking a project this large and with this much energy would indeed be a horrible solution. But if a guy like you who has some "street cred" is having difficulty getting access, then a new comer like myself is in dire straits even though Moodle is quickly becoming part of my job description.
I had hoped that I could hang out and continue submitting patches. Just getting patches accepted would be fine. But getting CVS access would make my life easier. But your story is not a story of hope.
If you are like me maintaing your own module in contrib, is not an acceptable solution. Forking a project this large and with this much energy would indeed be a horrible solution. But if a guy like you who has some "street cred" is having difficulty getting access, then a new comer like myself is in dire straits even though Moodle is quickly becoming part of my job description.
This all sounds.... well... pretty bad.
To top it off, I emailed Martin *again* but have yet to receive a reply *again*.
In addition, multiple peoples' opinions here are ones that I had strongly suspected myself, namely that:
a) forking something this huge is probably senseless.
b) maintaining a module in /contrib/ is not particularly useful, and in fact probably just makes my life more difficult in some ways.
Also, having a CVS installation for me isn't really an option, because our production web servers do not have a CVS client installed, and justifying the installation of a development tool on a production server would be something of a challenge. We have RCS on there for keeping backup copies of scripts and that's it.
So this is, I guess, one of those cases where "open source" and "open development" are polar opposites. Disappointing, really. I rarely come across open source projects that are designed in a sane enough way that I actually *want* to contribute to them.
To top it off, I emailed Martin *again* but have yet to receive a reply *again*.
In addition, multiple peoples' opinions here are ones that I had strongly suspected myself, namely that:
a) forking something this huge is probably senseless.
b) maintaining a module in /contrib/ is not particularly useful, and in fact probably just makes my life more difficult in some ways.
Also, having a CVS installation for me isn't really an option, because our production web servers do not have a CVS client installed, and justifying the installation of a development tool on a production server would be something of a challenge. We have RCS on there for keeping backup copies of scripts and that's it.
So this is, I guess, one of those cases where "open source" and "open development" are polar opposites. Disappointing, really. I rarely come across open source projects that are designed in a sane enough way that I actually *want* to contribute to them.
Well, I have just looked at the code. There are several security problems:
Other non security problems:
- missing access control - isteacher() should do the trick
- SQL injections - remember to quote all text fields of objects from db before executing update
- the use of execute() in runscript.php seems dangerous, why do you pass the directories through the $_POST? Let me guess, hmmm..., is it a trojan horse?
Other non security problems:
- missing postgresql support
- missing database scripts - see online assignment
- we are usualyydoing redirect after POST to prevent F5 problems, it solves some other problems too
- missing language packs
- lots of warnings in debug mode
Petr, was that reply meant to beat me down from wanting cvs access or something? You did not address CVS access in that reply at all, but all you did was criticize some shortcomings in my code, without pointing out that the module actually does things that people on the forums have been SCREAMING for for MONTHS, and that those screams have been 100% UNANSWERED by the moodle developers.
Hi Brian!
screaming is not the way. Preparing very clean code, being prepared to rework it following advise from core developers, and being prepared to wait is important. I would not insist with MartinD until you've followed Petr's advise.
And be prepared to sit on it for a while. Get many users to apply your patch and report bugs to you. And if the code is good, and the feature popular it will get there. Some of my feature work sits on my private branches for _months_... and I have CVS access! It's there, being used by a a handful of users who need it. Maturing like a good wine. And I do get criticism from other developers (Petr included) and I follow it to the last comma.
Moodle development is growing is size and speed, and we are moving to a dynamic where feature patches have to sit on "external" branches like yours for a while, maturing and waiting for a core developer to have a chance to review it.
It's not a new thing -- the linux kernel team does the same thing in a much larger scale... a lot of stuff that is in the kernel today circulated as unofficial patches for years, maturing.
screaming is not the way. Preparing very clean code, being prepared to rework it following advise from core developers, and being prepared to wait is important. I would not insist with MartinD until you've followed Petr's advise.
And be prepared to sit on it for a while. Get many users to apply your patch and report bugs to you. And if the code is good, and the feature popular it will get there. Some of my feature work sits on my private branches for _months_... and I have CVS access! It's there, being used by a a handful of users who need it. Maturing like a good wine. And I do get criticism from other developers (Petr included) and I follow it to the last comma.
Moodle development is growing is size and speed, and we are moving to a dynamic where feature patches have to sit on "external" branches like yours for a while, maturing and waiting for a core developer to have a chance to review it.
It's not a new thing -- the linux kernel team does the same thing in a much larger scale... a lot of stuff that is in the kernel today circulated as unofficial patches for years, maturing.
Brian, I am the one here who is bitching about insecure code, I am not good at talking or local politics. When I started some two years ago with Moodle my patches were overlooked and not accepted too (one of them was global searching, still one of the most requested features). I felt depressed too 
Then I got to know Eloy while reporting bugs in glossary. Eloy rewrote each of my patches, I have learned a lot, those were good times...
One day I saw Secunia advisory #12065, I said to myself: I will not go to bed till I find another security bug. My first report was a false alarm, hmm - really bad moment. It did not discourage me, after 2 days finally first major security bug found, yay - many more found later. I got my CVS access from Martin next month, I still got the email:
Martin Dougiamas wrote:
If you want to commit into main Moodle CVS you have to know both the Moodle code AND its developers. Before each commit I am asking myself:
On the other hand you can do whatever you want in contrib CVS. I do like the contrib freedom very much. It is very easy to get access, though it might take some time before martin or Eloy notices the request - the easies way is to flood the forum with long attachments with version updates Contrib is also very good for learning CVS basics. Seems there will be some improvements in contribs soon.
It takes a long time, before a module is moved from contrib into main. Some of them will stay there forever, such as my Book module. What are the benefits of working on contrib module?
Seems perfect? Of course we have problems too. The main one is the lack of documentation for developers and we do not have mentors that could guide new developers. I was very lucky, because Eloy and Martin helped me a lot in the beginning (thanks!). Also some modules do not have permanent maintainers - Chat, Assignment, Exercise, Workshop, etc.
Anyway keep coding, the people around Moodle are very nice and friendly community. I am sure they will appreciate your hard work.
skodak
One day I saw Secunia advisory #12065, I said to myself: I will not go to bed till I find another security bug. My first report was a false alarm, hmm - really bad moment. It did not discourage me, after 2 days finally first major security bug found, yay - many more found later. I got my CVS access from Martin next month, I still got the email:
Martin Dougiamas wrote:
Thanks for all your great help recently!At first I was afraid to commit any changes, I did not want to break anything. First I wrote Book module, then studied complete moodle codebase - each file except 3rd party libraries, I have found tens of security problems and even more normal bugs and only then started committing my own stuff into CVS.
I've given you (skodak) write access to all of Moodle.
Feel free to make small checkins with obvious fixes for bugs
like the ones you've been finding recently.
If there's anything more serious though, please talk about it
with me or Eloy first ... there's a lot of dependencies in a
project this size and seeing the bigger picture is difficult
for anyone.
Cheers and thanks!
Martin
If you want to commit into main Moodle CVS you have to know both the Moodle code AND its developers. Before each commit I am asking myself:
- is my code easy to understand/readable?
- is my code secure?
- would it break backwards compatibility?
- is this feature compatible with Martin's (or anybody else's) future plans? because once you add feature or even start implementing it in main CVS, you can not remove it - people start screaming in the forums at once
- does it hurt performance? there are for me unimaginably large sites out there, you should know a lot about both MySQL and PostgreSQL
- do I really need to modify database structure? Only Martin and Eloy do/approve changes AFAIK
- are my changes compatible with both STABLE and HEAD?
- etc., etc.
On the other hand you can do whatever you want in contrib CVS. I do like the contrib freedom very much. It is very easy to get access, though it might take some time before martin or Eloy notices the request - the easies way is to flood the forum with long attachments with version updates
It takes a long time, before a module is moved from contrib into main. Some of them will stay there forever, such as my Book module. What are the benefits of working on contrib module?
- freedom to try new things
- new coding skills
- knowing the insides of Moodle and its coding style
- becoming part of the community
- getting respect of other developers
- improving communication skills with potential users
- finally you might get CVS access to whole CVS
Seems perfect? Of course we have problems too. The main one is the lack of documentation for developers and we do not have mentors that could guide new developers. I was very lucky, because Eloy and Martin helped me a lot in the beginning (thanks!). Also some modules do not have permanent maintainers - Chat, Assignment, Exercise, Workshop, etc.
Anyway keep coding, the people around Moodle are very nice and friendly community. I am sure they will appreciate your hard work.
skodak
Thanks, Petr, and everyone else, for all of this information. Though I've been contributing small features and patches to open source projects for some time, it is also true that none have been as large as moodle in terms of either code base or number of developers, so I guess there are things that need addressing that I was ignorant of.
Thanks for helping me understand. I'll keep pluggin' away, and I hope I can figure out the right way to do some of this stuff. For example, Petr, you noticed I pass a directory name to an execute() call using $_POST. Do you have a link or advice on a better way of doing this? I'm not even sure that feature should be in there, but it's great for allowing computing students to upload files, and then testing to make sure they compile on the server (not just their local client).
brian.
Thanks for helping me understand. I'll keep pluggin' away, and I hope I can figure out the right way to do some of this stuff. For example, Petr, you noticed I pass a directory name to an execute() call using $_POST. Do you have a link or advice on a better way of doing this? I'm not even sure that feature should be in there, but it's great for allowing computing students to upload files, and then testing to make sure they compile on the server (not just their local client).
brian.
Petr, you noticed I pass a directory name to an execute() call using
$_POST. Do you have a link or advice on a better way of doing this?
You need to look at the required_param() and optional_param() methods in moodlelib.php. That is the safe mechanism that has been built into the Moodle core libraries for retrieving the data that the user sent you.
The last argument is a constant specifying what type of input you were expecting (for example an integer or a path name), and these routines check that what you've got really is one of those, and not, say, an attempt to inject malicous code into your site.
Probably the best way to find out how to use these functions is to look at other places in the code that already use them.
Hope that helps,
Tim.
You need to look at the required_param() and optional_param() methods in moodlelib.php. That is the safe mechanism that has been built into the Moodle core libraries for retrieving the data that the user sent you.
The last argument is a constant specifying what type of input you were expecting (for example an integer or a path name), and these routines check that what you've got really is one of those, and not, say, an attempt to inject malicous code into your site.
Probably the best way to find out how to use these functions is to look at other places in the code that already use them.
Hope that helps,
Tim.
I've found that CVS is very helpful even if it's just me coding on the stuff. Plus specially helpful when upgrading a Moodle code base that has custom changes in it, updating a "vendor branch" .
We have a Moodle installation with quite a few custom changes in it, and yes, it makes upgrading Moodle much more difficult than without them, but not impossible.
I don't see why the production servers would need a CVS client in your case...?
Update: Tracking_third-party_sources with CVS
Also, Martin D. has said that he gets around 500 mails per day (Edit 2: couldn't find the source on this, so I might remember incorrectly) + he has 1267 open bugs/feature requests assigned to him in the bug tracker, so this might feel like impenetrable at times...
We have a Moodle installation with quite a few custom changes in it, and yes, it makes upgrading Moodle much more difficult than without them, but not impossible.
I don't see why the production servers would need a CVS client in your case...?
Update: Tracking_third-party_sources with CVS
Also, Martin D. has said that he gets around 500 mails per day (Edit 2: couldn't find the source on this, so I might remember incorrectly) + he has 1267 open bugs/feature requests assigned to him in the bug tracker, so this might feel like impenetrable at times...
Wait, wait - it's not that bad!
Moodle is a lot easier to get involved with than many other open-source projects I work with. There is neccessarily a kind of "energy barrier" to making changes to the main codebase in a project like this, which makes the project manageable and reduces the likelihood of uncontrolled change, bugs, security holes etc. If the development access was completely open the project would fall apart.
The "contrib" CVS area provides a useful way to maintain serious code as part of the Moodle architecture, even before you get to the level of being a contributor to the main codebase. It also encourages modularity in your coding - this is a side-effect but I think it's a good one.
You don't need to fork the whole thing, as I've said. The CVS tool will maintain your local changes (you should make the changes/updates on your dev server(s) and then copy them to your production server(s) - surely you do that already?), and beyond that you should be able to keep track of things with a decent audit trail of your Moodle work.
Thanks, Dan. This makes some sense. I *do* already have a dev->production migration process. I misunderstood an earlier suggestion to do updates using cvs, which seemed odd to me, but it wasn't the first time I'd heard it either.
Of course, I would not assume that I would just automagically be given free reign over all parts of the moodle code. I merely wanted a way to make sure that my assignment type was as well integrated as possible, in the *most right* way possible. Getting it into CVS would, I hope, allow others with CVS access to help with some of the finer details, as Petr pointed out above, while I could do some of the grunt work of figuring out how new functionality could best be implemented within the moodle framework, and using the moodle API. It would also allow me a bit more freedom to add things like language packs, backup/restore code, and other things that would help me better integrate my module. These things aren't done right now, and the reason is because I don't want to touch any files outside of my module's directory.
I'll keep pluggin' along over here. Maybe if enough people find it useful and I get some help with things like when to use quotes so my users don't get befouled by attackers, and how to add postgresql support (I thought it was in the API and I didn't have to worry about it), then it will be deemed worthy by the powers that be someday.
PS - I appreciate your not assuming that I would plant a trojan horse in my code.
Of course, I would not assume that I would just automagically be given free reign over all parts of the moodle code. I merely wanted a way to make sure that my assignment type was as well integrated as possible, in the *most right* way possible. Getting it into CVS would, I hope, allow others with CVS access to help with some of the finer details, as Petr pointed out above, while I could do some of the grunt work of figuring out how new functionality could best be implemented within the moodle framework, and using the moodle API. It would also allow me a bit more freedom to add things like language packs, backup/restore code, and other things that would help me better integrate my module. These things aren't done right now, and the reason is because I don't want to touch any files outside of my module's directory.
I'll keep pluggin' along over here. Maybe if enough people find it useful and I get some help with things like when to use quotes so my users don't get befouled by attackers, and how to add postgresql support (I thought it was in the API and I didn't have to worry about it), then it will be deemed worthy by the powers that be someday.
PS - I appreciate your not assuming that I would plant a trojan horse in my code.
Getting it into CVS would, I hope, allow others with CVS access to help with some of the finer details,
We can definitely help you with advise before it gets into CVS. Code reviews and corresponding fixes always happen before new feature code is landed in HEAD. But in general you should still be the one fixing the code at this stage.
Hey, I don't think it's bad at all...
a) forking something this huge is probably senseless.
You're right. Don't fork it -- the Moodle team has a lot of people/talent and is moving forward all the time. You want to be on top of the moving train, not falling from it
What you are wanting to do, keeping local customisations, is called "tracking the vendor branch". Search for "vendor branch" in this forum and you'll find a very juicy thread. You can also google for that phrase but don't fall into the CVS/SVN trap -- even though their manuals have a section on tracking vendor branches, they are just awful at it.
Also, having a CVS installation for me isn't really an option, because our production web servers do not have a CVS client installed, and justifying the installation of a development tool on a production server would be something of a challenge
Well, we have very tight deployment mechanisms and still we have cvs and other SCMs in place because they are great as deployment tools. For a simple Moodle install with a small customization or two I just get an anon cvs checkout in place, apply the changes (developed and tested elsewhere) and then you can perform all sorts of queries, like "what have I changed locally" and "what files will be updated if I call cvs update now".
So, in short, cvs is much more than a dev tool. What we call version control systems are also known as "software configuration management" tools, and in that role they are invaluable on a server.
Martin,
I know you advise against SVN and I'd like to know what it's lacking for my vendor drop situation. I've made a decent amount of mods to my local Moodle install and applied changes to my initial vendor drop, then performed a few vendor drops and merged those changes from MOODLE_15_STABLE pretty easily (reading from http://svnbook.red-bean.com/en/1.1/ch07s05.html) So what is so bad about it?
Would I benefit and would Moodle benefit more (assuming my changes are ever used) if I switched to another SCM? I'd like to know specifically what the plusses are and then maybe I'll switch. (Hopefully by then there will be a TortoiseSVN-like program for a better SCM?)
Thanks in advance.
I know you advise against SVN and I'd like to know what it's lacking for my vendor drop situation. I've made a decent amount of mods to my local Moodle install and applied changes to my initial vendor drop, then performed a few vendor drops and merged those changes from MOODLE_15_STABLE pretty easily (reading from http://svnbook.red-bean.com/en/1.1/ch07s05.html) So what is so bad about it?
Would I benefit and would Moodle benefit more (assuming my changes are ever used) if I switched to another SCM? I'd like to know specifically what the plusses are and then maybe I'll switch. (Hopefully by then there will be a TortoiseSVN-like program for a better SCM?)
Thanks in advance.
For very small changes, and specially situations where you don't have much in terms of conflicts, CVS/SVN are kind-of alright. You still have to deal with -k mode conflicts.
But it very quickly breaks down when you
- find a bug
- fix it in your SVN copy
- submit it to the bugtracker or a core developer and get the fix into official CVS
- update your 'vendor branch' and merge with your local changes
- Conflicts because your bug has been echoed back to you. Yuck!
It's trivial with 1 bugfix affecting a few lines. But multiply that for a couple hundred and add feature work happening on your local branch that goes into official CVS and gets echoed back.
Merges then result in enourmous conflicts that are mostly trivial, but in their triviality they mask small fixes -- I can't discard the conflict in file X which where I know I made a fix because, someone may fixed a really small typo or subtle thing.
So in a sense, after people use SVN/CVS for this, they learn to not send patches upstream (we are all a bit pavlovian), which is the wrong thing to learn!
I probably don't understand how the CVS tree really works. My most immediate problem with it is just that it doesn't look to me like it would offer much benefit in terms of ease of development. What I mean by "ease of development" here is the ability to add features to Moodle that make sense in the most generic sense, and at the same time enable the addition of features to something like a contrib module or (in my case) a new assignment type.
I'm going to set up a local CVS repository this week and look to see how this would work, and how I would track vendor changes (never had to do that in cvs) and how this would all work with our production moodle site, and the code I've written to add functionality. Hopefully I'll figure out a solution.
I'm going to set up a local CVS repository this week and look to see how this would work, and how I would track vendor changes (never had to do that in cvs) and how this would all work with our production moodle site, and the code I've written to add functionality. Hopefully I'll figure out a solution.
Jumping on the bandwagon here since I'm in the same boat as well. I sent a msg to Martin D. a while ago about contrib access and assume he was too swamped to answer. So if anyone comes around handing contrib access out, I'll take one. ;)
I've been working on parental access and some additions to the Linuxbox gradebook, which Michael Penney is now working on as well. I want to get this into contrib (so it's in a centralized place instead of my own cvs server where it is now) so the bigwigs can check it out, give suggestions, and see if they'd like to put it upstream since a lot of people have requested these features as well.
I've worked on Drupal before and the bug tracker/feature request system and the forums are integrated and I feel that is a reason for much of Drupal's success and rapid growth. It's very easy to submit patches, get them reviewed and applied to certain sections/modules of Drupal. Moodle's bug tracker is so separated from the forums and community it makes it very sluggish and inefficient to get fixes applied.
I've been working on parental access and some additions to the Linuxbox gradebook, which Michael Penney is now working on as well. I want to get this into contrib (so it's in a centralized place instead of my own cvs server where it is now) so the bigwigs can check it out, give suggestions, and see if they'd like to put it upstream since a lot of people have requested these features as well.
I've worked on Drupal before and the bug tracker/feature request system and the forums are integrated and I feel that is a reason for much of Drupal's success and rapid growth. It's very easy to submit patches, get them reviewed and applied to certain sections/modules of Drupal. Moodle's bug tracker is so separated from the forums and community it makes it very sluggish and inefficient to get fixes applied.
Hi, Brian!
Which address did you use to send me mail? I never saw it, but this is not uncommon because I simply get too much to read (and also lead any kind of normal life at the same time). These days I don't read martin@dougiamas.com or martin@moodle.org AT ALL.
The best way to reach me about CVS access is to follow the directive on the newish CVS documentation and use the moodle.com helpdesk. You could also use the messaging tool here in moodle.org or Skype me.
This is not an open invitation for everyone to start spamming me that way, but for real project-related stuff, that's what works.
Cheers,
Martin
Which address did you use to send me mail? I never saw it, but this is not uncommon because I simply get too much to read (and also lead any kind of normal life at the same time). These days I don't read martin@dougiamas.com or martin@moodle.org AT ALL.
The best way to reach me about CVS access is to follow the directive on the newish CVS documentation and use the moodle.com helpdesk. You could also use the messaging tool here in moodle.org or Skype me.
This is not an open invitation for everyone to start spamming me that way, but for real project-related stuff, that's what works.
Cheers,
Martin
Hi Martin,
Thanks for the tips! I had seen the cvs document, but the intro to that section of the cvs document seems to imply that you've already received an invitation to join the moodle development team, which I haven't, so I didn't think that section was speaking to me. I've just registered at the helpdesk (I thought I was already registered, because I thought that's where I submitted bugs and feature requests before). Once I get the confirmation, I'll be on my way.
Thanks!
brian.
Thanks for the tips! I had seen the cvs document, but the intro to that section of the cvs document seems to imply that you've already received an invitation to join the moodle development team, which I haven't, so I didn't think that section was speaking to me. I've just registered at the helpdesk (I thought I was already registered, because I thought that's where I submitted bugs and feature requests before). Once I get the confirmation, I'll be on my way.
Thanks!
brian.