Moodle in English: Moving from CVS to git in November 2010

Moving from CVS to git in November 2010

by Martin Dougiamas - Monday, 22 November 2010, 5:58 PM

Hi all,

This is just a heads-up to let you know that we at Moodle HQ are working on moving to git.moodle.org as our main server soon, at which point cvs.moodle.org will become a read-only mirror of it. As a result, you may lose CVS access if you had it and if you want to continue working on Moodle code then learning to use git would be a good idea.

At the same time we are going to be reworking our processes to take more advantage of the flexibility that git gives us. This means a lot more reviewing of code and STABLE branches that are even more stable. More about this soon, and come to the developer meeting on Thursday to talk with us about it!

Average of ratings: Useful (2)

Re: Moving from CVS to git in November 2010

by Joseph Rézeau - Monday, 22 November 2010, 6:20 PM

"... As a result, you may lose CVS access if you had it ..."

What does/will one have to do in order to "transfer" one's current CVS access (username and password) fromo CVS to GIT?

Joseph

Average of ratings: Useful (1)

Re: Moving from CVS to git in November 2010

by Dan Marsden - Tuesday, 23 November 2010, 4:26 AM

depends on the process decided by HQ - there are a lot of different methods available using git and I expect the commit workflow to change - see Martins comment above

"This means a lot more reviewing of code and STABLE branches that are even more stable."

improving reviewing could mean less people have access to commit to core - I for one would be happy to "give up" my core access if it meant improved reviewing processes (as long as reviews are performed in a timely manner!)

..but discussion on this right now is a bit pre-mature - I'll be attending the meeting to hear what Martin/HQ have planned - I'm sure there will be the opportunity to discuss the future plans in the meeting!

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Petr Skoda - Tuesday, 23 November 2010, 10:10 AM

Hello,

exactly - timely review is the critical factor here. In any case the workflows will be most probably evolving. We can not plan everything in advance, the point is to start using git and see what works for all of us best.

If you do not know much about git please start learning it and using it for some small projects. Once you start using git suddenly any migration plan will seem trivial, no matter what model we end up using.

My personal wish list is:
* easier cooperation of developers
* fewer regressions in stable
* fewer unfinished features
* higher code quality

Petr

Average of ratings: Useful (1)

Re: Moving from CVS to git in November 2010

by Oleg Sychev - Wednesday, 24 November 2010, 9:58 PM

Hmm, yes timely reviews (stressing TIMELY) would be quite good and somewhat new experience with Moodle if there are any chances of them. Thought I guess that new VCS doesn't add more time to developers who could do a review. Still I hope DVCS at least will make it much easier to maintain patches for these 1-3 years while they would be waiting for timely review. ;(

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Howard Miller - Tuesday, 23 November 2010, 10:30 PM

I'd be more than happy to loose direct commit access. I'd much rather any stuff I did was double checked so it was somebody else's fault when it all goes horribly wrong

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Juan Leyva - Tuesday, 23 November 2010, 8:18 PM

Hi all,

Contrib code will also be hosted in GIT?

There are a lot of people with access to /contrib/plugins

Will be need to request a new user account in git for contrib dir?

Regards

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Petr Skoda - Tuesday, 23 November 2010, 9:18 PM

Hi!

This change is not related to contrib plugins at all, developers will be able to apply for CVS access to contrib even after the git migration is completed. It is still highly recommended to apply for contrib CVS access, register a plugin name and commit release snapshots to CVS regularly.

You can start using git for development of contrib plugins now. Nobody is going to force you to do so in the future, if you really really want to endure the pain of using CVS for development purposes feel free to do that.

We need to create new documentation and guides for contributors, hopefully this should be done sometime next week. The details should be discussed during the dev meeting later this week, hopefully there will be some volunteers that can help with writing of these new git guides for Moodle.

I have started to update our git docs page at http://docs.moodle.org/en/Git - this should serve as the starting page for developers and administrators. A complete list of git related pages will be available at http://docs.moodle.org/en/Category:Git

Petr

Average of ratings: Useful (3)

Re: Moving from CVS to git in November 2010

by Dan Marsden - Wednesday, 24 November 2010, 4:34 AM

Some developers have already started moving their Contrib stuff into git - see:

http://github.com/skodak/

or mine: https://github.com/danmarsden

I found a cool way of generating a patch in github - which works really well for plugins that make changes to core code - you keep a copy of the standard moodle branch in your repo, use a seperate branch for your changes, then use the branch list page to compare 2 branches in your repo:
https://github.com/danmarsden/moodle19-patch_turnitin/branches

gets this:
https://github.com/danmarsden/moodle19-patch_turnitin/compare/MOODLE_19_STABLE...mdl19-turnitin

then you can add ".diff" or ".patch" to the end of the url to get patch/diff:
https://github.com/danmarsden/moodle19-patch_turnitin/compare/MOODLE_19_STABLE...mdl19-turnitin.diff
https://github.com/danmarsden/moodle19-patch_turnitin/compare/MOODLE_19_STABLE...mdl19-turnitin.patch

..I thought it was cool anyway...

Average of ratings: Useful (3)

Re: Moving from CVS to git in November 2010

by Myles Carrick - Wednesday, 24 November 2010, 6:52 AM

Great tip, Dan. One thing that's changed the workflow too for many open source projects on github is the pull request mechanism: http://help.github.com/pull-requests/ - it makes it really easy for contributors and curators of projects.

I'm stoked that the project is now going to git. The Ruby on Rails project was for its first few years _much_ smaller than the Moodle community. Check out this visualization of its commit history before and after the move from SVN to Git and GitHub 2.5 years ago (April '08):
My experience in that project too is that the massive growth in "core" activity is actually tiny in comparison to the growth in plugins and libraries around the project. It would be truly awesome if we see something similar with Moodle

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Juan Leyva - Wednesday, 24 November 2010, 4:58 PM

Very interesting Dan,

In your turtinin plugin for Moodle 2.0 the download link goes to a zipball in github

This zipball is created automaticaly every night, x hours...?

Regards

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Dan Marsden - Wednesday, 24 November 2010, 5:43 PM

it's generated on the fly - so it's always up to date.

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Tim Hunt - Thursday, 9 December 2010, 12:49 AM

If you have not been following these things, Development:Process is shaping up quite nicely.

At the moment, they seem to be focussing on the internal procedures for Moodle HQ staff. That is fair enough. I would like more details on how external developers and component maintainers fit it, but I am sure we will get that it due course.

The bit I wanted to raise is what exactly needs to be included when you issue a pull request, asking someone to review your code. This could be external developer -> component maintainer, hq developer ou -> integration reviewer.

Of course, the most important thing is the code you are submitting, and other infomation should be in the associated bug report. At the OU we use the 'Handover checklist' that I included below. When you think you have finished something, you copy and paste the checklist into a comment in the bug report, and then edit it to give the appropriate answer to each question, and then you get someone to review the code.

I actually quite like this, which is a bit surprising, because in general I hate overly bureaucratic procedures. However, there have been enough times where the checklist has reminded me to run the unit tests, or check backup and restore, when something was broken that I think it is worth the few minutes it takes for me to fill it in.

Obviously, this checklist is overkill for stimple bug fixes, and we don't use it for that. It is mainly for new features. Anyway, I just thougth I would share our checklist here, so that Martin and friends can consider something similar for Moodle development.

HANDOVER CHECKLIST
------------------

Specification document:

N/A
[add URL for document here if there is one, leave n/a if there isn't or it is superceded by the functionality document]

Functionality document:

[add URL for document here or, for very simple features, write a few paragraphs here]

Feature completion checklist:

1. Support for groups/groupings

[Sample answers:
Y - Tested, including both groups and groupings.
N/A - groups cannot apply to this development.
N/A - groups could be implemented but we have agreed they are not needed.
]

2. Backup/restore

[Sample answers:
Y - Tested.
N/A - no course-related data in database or filesystem.
N/A - data is only stored in existing backed-up locations.
N/A - data does not need to be backed up.
]

3. Capabilities

[Sample answers:
Y - The capabilities (described in functionality document) work correctly and can handle OU requirements.
N/A - no user-interface.
N/A - no functionality.
]

4. Accessibility

[Sample answers:
Y - Basic developer accessibility check done (including issues in the Pre-evaluation accessibility checks document).
N/A - no user-interface.
N/A - no changes to HTML.
]

5. Browser support

[Sample answers:
Y - Tested in Firefox and IE7. Appearance and functionality OK, no browser errors.
Y - Tested in Firefox, IE7, and IE8. Appearance and functionality OK, no browser errors.
N/A - No changes to HTML.
]

6. Validation

[Sample answers:
Y - All changed HTML pages were tested with the W3C HTML Validator and show green with no errors (either in the new code or anywhere else).
N/A - No changes to HTML.
]

7. Install and upgrade

[Sample answers:
Y - Tested fresh Moodle install and upgrade from existing install.
N/A - no database structure changes.
]

8. Performance

[Sample answers:
Y - The code adds [insert number] database queries to [insert page name], which an LTD has agreed is appropriate.
Y - The code creates new pages. For [a system with 10,000 courses / a course with 10,000 users / a quiz with 100 questions / other metric of similar size relevant to the work the page does] these pages use up to around [insert number] queries, which an LTD has agreed is appropriate.
N/A - The code does not add any database queries.
]

9. Unit tests

[Sample answers:
N/A - There are no unit tests in this existing code area.
N/A - I couldn't be bothered to write unit tests for this new code area.
Y - I wrote new unit tests. These run with no failures. Give me a gold star!
Y - There are existing unit tests. These all still pass.
Ish - There are existing unit tests. These run with no more failures than before.
]

10. Impact on other platforms (VLE, OpenLearn, VITAL, CPLD etc)

[Sample answers:
Y - This feature is desired on all platforms.
Y - This feature has an 'on/off switch' as part of its standard Moodle settings.
Y - After consultation with a relevant LTD, we agreed how to disable the feature on [platforms].
N/A - This existing feature, now being amended, is not used on other platforms.
]

11. Admin settings / necessary configuration

[Sample answers:
Y - I have added an entry to the release update bug which tells the system team the correct admin settings to apply during the upgrade (for both acct and live servers).
N/A - This feature has admin settings but defaults are correct for learn, learnacct, and other platforms
N/A - This feature has no admin settings.
]

12. Completeness

[Sample answers:
Y - Feature is functionally complete and ready for release.
]

13. Wiki page

[Sample answers:
Y - I have added a short section about this feature on the release features wiki page for the relevant version.
]

Release notes:

[Any known problems]

Average of ratings: Useful (1)

Re: Moving from CVS to git in November 2010

by Tim Hunt - Thursday, 9 December 2010, 12:53 AM

I should add, some of that is clearly OU-specific and not relevant to Moodle. E.g. replace "Wiki page" with "Release notes for the next version".

Also, something that is not in the checklist, but which would be good to add. We now that the QA_testing procedure for major releases. I think that worked quite well for the Moodle 2.0 release. Obviously, whenever a new feature is added to Moodle, we will need some new QA tests help the testers to verify it works. So I would propose adding a "New QA tests" section to the checklist, to remind people to supply those.

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Oleg Sychev - Thursday, 9 December 2010, 10:46 PM

Tim wrote "If you have not been following these things, Development:Process is shaping up quite nicely."

Reading...(there should be a smile, depicturing utter shock) Are you, guys, serious about that?! Really? 3 stages to stop changes to be integrated (later ones clearly understaffed) and no one to push them throught?!

You have now much simpler system with QA analysis (which could be done by much broader range of people and don't prevent code from been in Moodle code base). But even it is not working. Some quick stats I gathered using tracker from all issue I reported (mostly bugs, fixed by moodle core developers with rare reviewed and checked in patch from me). 39 issues are Resolved as Fixed, but don't Closed (i.e. not been QA reviewed). And 12 ones that is Closed. Of those 12 only 7 was closed by QA reviewers, 5 was closed by the developers who fixed the issue. Total 7/51 = 14% of fixed issues been throught QA Review process.

And now you tell us there will be 2 stages of QA (3 people on the first, 2 on the second) after the component maintainer pull you code (which is hard enought to achieve as it is, since they are busy people), and no way around them? How many time they a plan to spend reviewing other peoples code during week? Unless they are resolved to do reviewing most of the time (I quite doubt that this is the case, since they are good at writing code and probably like it), even component maintainers would have problem pushing their code to the production repository. External developers? Whom are you kidding? (someone said "timely review"? ;).

P.S. Tim, thanks for the check list. It is useful as a template for my own one.

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Pierre Pichet - Saturday, 11 December 2010, 6:27 PM

Hi Oleg,

I had the same reaction about the multi-level code reviewing.

We can however note that this change is proposed as Moodle 2,0 has been released ( more or less beta version...)

so there is less urgency in important code modifications.

The Moodle beast has grown to more than 1 million code lines including the various librairies.

Developping 2,0 has been a much longer process than expected and important issues like backup and restore were solved at the last minutes.

Some peoples are already thinking of 2,1 which will be at least a many months project.(i.e. Tim question-engine ).

The idea seems to trade time vs better code reviewing .

Using GIT will change the way things are developped and direct and instaneous access to HEAD will not be the way to build new things.

Most of the development work will be off-line and each sub project manager (i.e Tim) will be able to set the necessary write access

to the sub project.

What I understand also from some CHATs is that separate GIT HUB could be set even for each bug tracker issue.

Let's play the game.

Pierre

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Oleg Sychev - Monday, 13 December 2010, 5:15 AM

Hi, Pierre.

"The idea seems to trade time vs better code reviewing.", more precisely, the idea seems to trade amount of new code (features, fixes etc) vs better code reviewing. Time is not the case, since there are no queue of changes - some other changes could bypass delayed one, and the process may continue on indefinitely. And if reviewers are going to spend significant time writing their own code, that quite a number of changes (fixes includes) would be stalled (except their own code, of course). If reviewer let bad code pass, it's his fault, but if he doesn't review code - he's not bothered by it, so why should he strive to review more code?

I'm looking on Moodle from several points: as a teacher, using it in my courses, an admin of university installation and an external developer. Even been part-time admin (looking to spend as small time as possible administring moodle), I'm quite content with its codebase quality. For now I see that quality of code isn't the change Moodle failed most (there are also easier ways to increase this quality, which no one seems wanting to implement, that's really strange), but the challenge of integrating all good code, provided by community.

GIT would definitely help external developers to maintain their patches without been reviewed and checked in into Moodle - rebase alone worth a lot. But that is good only in one branch: upgrading them to the next one would be a nightmare. If the Moodle would fail to integrate most of community-generated code of decent quality, the result would be a multitude of alternative GIT repositories, which features Moodle with several advancements, and a users completely confused which one to use. If Moodle would be lucky, it may end up in Linux-like mode, with several important distros and a core that no one uses alone.

"Let's play the game." I'm afraid I am too tired of been on the reserve bench in this game, sorry. I'm completing several moodle-related projects with my students, and we'll post the code on the tracker if it's good enought - but that is merely inertia. I don't really beleive it would be reviewed and checked in. After getting only 2 small patches reviewed in a year with many more rotting on the tracker without any attention (except words about how this is good and useful thing). I was impatient to get reviewing early on - now I'm not. And I already added to my TODO list an entry to look for another open-source project I could work on with my students in next fall semester, the life should go on. It seems I should resort to plugin-only mode with Moodle. I (as many others) already was forced to fork assignment, thought I don't like it. I hate been forced to fork quiz/questions. But I see no way around it.

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Pierre Pichet - Sunday, 19 December 2010, 1:43 AM

"TODO list an entry to look for another open-source project I could work on with my students in next fall semester,"

We are still in december and the new GIT reviewing process is not really started, let's give a chance to progress.

I am as concerned as you are about the efficiency of the reviewing process which will first feature only 4 persons to have final write access...

Somewhere either they found a new generation of Supermans ( or a code specific tea infusion) or they will come back with a more efficient reviewing structure.

As main 2,0 code is already released, this is a good time to test a new structure, so I repeat Let's play the game. but be ready to give the necessary feedback for improving the process.

I am quite confident about your ability to feedback

Pierre

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Oleg Sychev - Sunday, 19 December 2010, 3:01 AM

Pierre, you may misunderstand my phrase. I'm not going to switch Moodle as our LMS, I was talking about a developing work I am doing with late-year students on the Moodle code. The students may get really good experience working on real open-source project with good developers, I am able to orchestrate larger code than by myself, and the quality of posted there code is probably better than my own, since supervising other people you are usually more strict than doing that to youself (because you don't concentrates much on writing it, working in pair teacher-student gives less errors than just teacher coding).

But all this is good if there are at least some reaction from given open-source project to the posted code. There were almost none. Most code we written in 2009 autumn still sits there waiting reviewing with no reaction at all - and there were neither GIT, nor new process aroung all the time. One small patch from this autumn you know of is stalled too. That's not the thing, that would encourage students working on open source at all. So - for the sake of teaching - I must look for a better occasions. If I may wait for years, the student's won't - people writing code 2009 autumn already graduated and leave university. Time matters. I'd prefer to work on Moodle since it's improvment help me to teach better, but having a good code for Moodle is a byproduct of teaching there, not the goal.

And "look for" doesn't necessary means I'll find it right away:I don't think there are dozens of good projects with core devs and code base as good as Moodle just waiting for our input. But I must look for a better alternatives still, so I could choose next autumn. I don't say I won't look at Moodle and see, if things are improved. Tracker will send me messages if it happens.

"Somewhere either they found a new generation of Supermans ( or a code specific tea infusion) or they will come back with a more efficient reviewing structure." or they shortcircuit development on youself and very limited number of people they know (just like Tim shortcircuit quiz development on him and you). What would prevent them from doing that?

"As main 2,0 code is already released, this is a good time to test a new structure," I disagree. Moodle 2.0 has developed too long without real-word testing, so it has a high chances of repeating 1.7 problem, behaving strange in real-word situations. Now we may end up with all those strangeness and inconvenience at hand and many obstacles before fixing them.

"I am quite confident about your ability to feedback" one person, who knows me, once said that I usually give good feedback - as a parting gift.

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Pierre Pichet - Sunday, 19 December 2010, 5:09 AM

As I wrote : I am quite confident about your ability to feedback

so thanks for your supplementary comments.

Pierre

P.S. Not being a professional programmer but an analytical chemistry teacher who need flexible quantitative questions, I am quite at ease that Tim has the final word on my code proposals given his high competence and involvement to Moodle.

Average of ratings: -

Re: Moving from CVS to git in November 2010

by Oleg Sychev - Sunday, 19 December 2010, 8:14 AM

Pierre, I already told there I too prefer Tim's final word. He is better programmer and could hint better solution or potential problem. But how do you feel youself receiving these word for only 10-20 lines of code in a year?! That's not a joke, that's my experience. Is there a point then to write it at all?

P.S. I am working on some projects that would contain code, that may be using in you quantative questions. Would you like an expression parser for one thing?

Average of ratings: -

Re: Moving from CVS to git this week!

by Martin Dougiamas - Monday, 13 December 2010, 3:02 PM

Just an update on this.

it's a little later than expected, but we are definitely switching to git this week.

All direct CVS access will be switched off today, and during this week CVS will be converted to a read-only copy of the main repository git://git.moodle.org/moodle.git (which will have write access enabled to it).

If you are a third party and have code that you want to get into Moodle, then make sure there is an issue in the tracker addressing and describing it, and then do one of these:

if the component has a maintainer, submit it as a patch or git url to the issue so the component maintainer can deal with it
if there is no component maintainer, or you ARE the component maintainer, initiate a pull request to have your code reviewed for inclusion in Moodle. You do this by creating a new issue here: http://tracker.moodle.org/browse/PULL

Average of ratings: Useful (2)

Re: Moving from CVS to git this week!

by Tim Hunt - Monday, 13 December 2010, 5:21 PM

Re 1. If someone has a quiz patch that they want reviewed by me, can't they also file a pull request at http://tracker.moodle.org/browse/PULL, and just assign it to me, rather that the main integration team?

That way, when I have reviewed the change, I can just add a comment and re-assign to the main integration team.

Average of ratings: -

Re: Moving from CVS to git this week!

by Martin Dougiamas - Monday, 13 December 2010, 6:15 PM

The trouble with letting anyone file pull requests and assign them at the moment is to avoid having to triage lots of bad pull requests and waste the integration reviewers time (they have to be timely!)

The PULL project in the tracker is a first pass at making something usable to get Moodle HQ going without impacting much on the main tracker.

But I agree with you, it would be good to have the same tools available for component maintainers. Still working on this. It may be better to include all this in the main workflow for the original issues - we just need to experiment more with configurations.

Average of ratings: -

Re: Moving from CVS to git this week!

by Dan Poltawski - Wednesday, 15 December 2010, 5:56 AM

I think I like this concept, working to make it more integrated and available for 'component maintainers' seems a definite win for me (ideally i'd like to see this work outside of jira and somehow with git, as that seems relatively high overhead for myself to request a 1 line fix pull, for example).

I'd also quite like the concept of making myself available to external contributors to pull from using the same sort of workflow even if its not in my component. (Basically so I could make myself available as a 'puller' from which you could get reviews from and I could send my pull onto the component maintainer (with maybe a thumbs up from me)).

Average of ratings: -

Re: Moving from CVS to git this week!

by Petr Skoda - Monday, 13 December 2010, 6:37 PM

No, it is not possible. Developers should submit only high quality/reviewed patches to the PULL project - the same code you would previously put directly into CVS. Our Jira PULL project is a replacement for CVS commits and weekly review process.

I understand that you may not like the extra work of making pull request yourself for all the changes in quiz and questions because it is extra work compered to previous CVS workflow where anybody could commit anything without any review. We are starting with a small group of people that can submit the pull requests (component leads) because we need to tune the processes first. Unfortunately there are currently not many people that would be doing the code reviews before the pull request - we need a lot more man power there.

In the future we should of course improve the Jira workflow in the MDL project so that it makes life easier for component maintainers. But now we need to migrate to git and start the new processes asap, redesigning and reconfiguring the MDL Jira project might take months and we need to cleanup the issues there first.

Petr

Average of ratings: -

Re: Moving from CVS to git this week!

by Oleg Sychev - Monday, 13 December 2010, 11:11 PM

Could we starting improving workflow in Jira by adding a new custom field, that would allows us clearly mark, which issues have attached patches from external developers, and which have them from core developers?

It would be a lot easy for the component maintainer to filter out patches (or something), that require his attention (from general bugs/feature requests) this way.

P.S. I guess component maintainer could MOVE issue into a PULL project after he reviewed it.

Average of ratings: -

Re: Moving from CVS to git this week!

by Davo Smith - Tuesday, 14 December 2010, 12:37 AM

I was just about to suggest a standardised tag to achieve that, e.g. 'external-patch'.

However, that would then require educating everyone to use that consistently...

Average of ratings: -

Re: Moving from CVS to git this week!

by Oleg Sychev - Wednesday, 15 December 2010, 9:46 AM

Well, Davo, I guess this thread is a measure of how much attention would get external developers. Even when it comes to such simple thing as additional field in the tracker (which will help most component maintainers). There are practically two separate discussions. The Big Guys solve their problems, they have no time (or inclination) to react on such things.

Average of ratings: -

Re: Moving from CVS to git this week!

by Tim Hunt - Wednesday, 15 December 2010, 3:50 PM

But it is not clear that field in the tracker really solves the problem we all know is there. They are actively working on these questions of workflow right now, and yes they are starting with the procedures for the in-house staff, but I don't see a problem with that. Seems like a logical order to do things in.

Average of ratings: -

Re: Moving from CVS to git this week!

by Oleg Sychev - Wednesday, 15 December 2010, 6:07 PM

Well, Tim, did you want to be able filter out issues with patches from these without them? It didn't solve problem - I guess OU would not drive you less hard from that field. But this probably can make you job easier when looking for stuff. (maybe you would found find some patch later when you'd have more time and there would be no me already ). And additional search argument won't hurt much. It isn't hard to add either.

Average of ratings: -

Re: Moving from CVS to git this week!

by Tim Hunt - Wednesday, 15 December 2010, 9:53 PM

I would definitely like to be able to see cearly which patches I have outstanding for review. And I expect that once the HQ people have finished setting up git, and the tracker, and the new processes, we will have that. In the mean time, I will be patient, or make helpful suggestions. But I won't expect everything to be sorted out immediately.

Average of ratings: -

Re: Moving from CVS to git this week!

by Oleg Sychev - Friday, 17 December 2010, 4:43 AM

Tim, would you call me impatient? How many months ago I reminded you last time about one of my patches, waiting for you review? I guess this was July.

And I don't see any reasons to be patient, sorry. Does Moodle development process rewards patience from external developers? Never heard of such thing. Experience with Moodle devs tell me that when you getting impatient, you have small (about 10% maybe) chance to get something moved forward. I don't remember single event in years when I patiently sit and wait and something happened (no offence, just observation). Even Developer Docs suggests trying impatience for external devs

So I don't have much expectation anything to happen - actually you reacted on my post, when I tell Davo that we shoudn't expect anything in this area - but I see no reason to behave otherwise too. It's you who expect this to happen sometime at all. I am not.

"And I expect that once the HQ people have finished setting up git, and the tracker, and the new processes, we will have that." Who are you kidding? Me or youself? Maybe you could name any event in last years of Moodle history when Moodle HQ sorted out their problems and turned to external devs? Again, experience tells that when they get these processes sorted out, they - most likely - will be so anxious to finally made 2.1 release and fix at least those 2.0 sillyness, that would anger users most, that external devs matter would be left to wait some more happy time (which, as we all know, never happens). So now is as good (or as bad) time as any other.

Average of ratings: -

Re: Moving from CVS to git this week!

by Dan Marsden - Friday, 17 December 2010, 6:22 AM

"turned to external devs"

couple of examples:

Scorm - Piers and myself
Portfolio API - Penny
setting up initial read-only git repo - a few of us from Catalyst

This is something HQ does a lot! - there are a lot of other examples - probably too many to list.

Average of ratings: -

Re: Moving from CVS to git this week!

by Oleg Sychev - Friday, 17 December 2010, 7:35 AM

Dan, you are telling about a company of external devs, that is an official Moodle Partner. It's quite not the same thing as community of individual developers, not backed up by a large university like OU.

Average of ratings: -

Re: Moving from CVS to git this week!

by Dan Marsden - Friday, 17 December 2010, 8:04 AM

ok - those examples were partner specific, but HQ looks for people with the experience and knowledge to do the job best - it just happens that the examples I gave above were from Moodle Partners - for example HQ tried to source external help for SCORM 1.2 several times (not using devs from a Moodle Partner) but was unsuccesful finding someone who could follow through with the work - in the end it was us who had the skills that were able to follow through and complete the SCORM 1.2 work.

Of course, now that the number of Partners who employ senior Moodle developers has increased what better resource for HQ to tap into for extra support?

There are many other examples where small bits of of development/patches have been written by external developers (I see a lot of SCORM fixes like this!) - it just takes time for me to get through all the issues and review them, as it does for other component maintainers like Tim who have an even bigger buglist!

Of course...if you really really want something to end up in core - the best way would be to pay a core dev to review your code... otherwise patience and (gentle) persistence are probably the best way forward.

re-reading the posts, I don't think the new changes to the tracker process/git will negatively affect people like yourself - the reporting process is still the same, the changes really only affect component maintainers/users with write access to cvs - You it will still require a core dev/component maintainer to review your patch, but this may even happen quicker with the new internal HQ processes!

I suggest you give a little bit of time to see if things improve - we all know that some bugs sit for longer than they should, HQ is trying to improve this by putting new systems in place - lets give them some time to iron out the process.

Average of ratings: -

Re: Moving from CVS to git this week!

by Oleg Sychev - Friday, 17 December 2010, 7:50 PM

Dan, you are just backing up muy original statement that Moodle Big Guys doesn't care much of simple users, their little problems and patches for now

See, all you examples are about things HQ wants to be done, while community usually comes with solutions to their own petty problems. Getting SCORM certificate is one thing (shiny thing, good to put on corporation portfolio), reducing the number of clicks (or page loads)to do some particular frequent operation from O(N) to O(1) is quite another one (and neither developers, nor people making decisions in big universities are affected by this much). - but that is another things for people who do regualr work and deal with this on the daily basis.

All that is not necessarily bad for Moodle itself (I am no one who decides what is good for Moodle). Maybe convenience for the regular users is not a priority at all..But once it's was these simple teachers who made Moodle so widespread, and losing them it may lose some of it's natural strenghts. I have seen a paper comparing Moodle with other CMS's several years old, and concluding that Moodle wins more because it's a teacher solution, not institutional one. Now it drifting to the institutional solution class. Again, that may be not bad - but if during this it's ceases to be a teachers solution, that is sad.

Re-stating about the changes to the process, GIT will help exteranl developers much. The process may hinder them. Consider following scenario: you wait some unspecified time (months or years from experience, upgrading you code according to the Moodle development) to get component maintainer to review and apply you bit of code. Than you wait some unspecified time until this change is reviewed by second QA team. Then it's get rejected on that review and you are told to rewrite it. That you again wait several years more for attention of component maintainer etc. As Nasreddin tolds in a popular saying abouit teaching a donkey to speak, for that term either donkey will die, or the Shah who ordered it will die, or I myself will die. ;) OK, that is worst case scenario, but not quite unlikely one, and developing every process you should be aware of them.

Average of ratings: -

Re: Moving from CVS to git this week!

by sam marshall - Friday, 17 December 2010, 9:34 PM

Hi, just by the way, I agree with you on certain points:

- If an external developer that is not close to the core dev team wants to make a change in core behaviour, this is very very very unlikely to ever happen. If the change is only a bugfix it is still fairly unlikely unless the bug is very serious, the fix is very trivial, etc...

- With regard to changes, part of this is because the chance that said external developer would make the right change first time is very very (one less...) unlikely to ever happen.

- The other part is because core developers do not usually have time to review patches that aren't in line with a particular existing objective. Also it is pretty pointless submitting a patch that changes something if that hasn't been agreed already in advance with core developers that changing that thing (in that way) is a good idea, but again, core developers are not necessarily available to respond to that kind of suggestion.

- Martin has said several times that for 2.1 he wants core developers to be more receptive, ie time specifically allocated, to reviewing external changes. I agree that there is every reason to be sceptical about whether this will actually happen.

However there are other points you are not quite right about:

- This is also very (down to a single 'very' this time) difficult for a major university like the OU - outside of the quiz engine which we directly maintain, it's been extremely difficult for us to make changes in core, partly for some of the same reasons. I think things have been getting noticeably better recently though!

- We have a quite large development team here but in order to try to get changes into core, only a few developers who are very familiar with it are coding those changes. I.e. we don't think core will accept our 'standard OU quality code', it has to be higher quality and particularly, produced with more understanding of Moodle (than our other developers have, who are still professional developers and who work on our moodle-based system all of the time). That's a tough quality barrier for an independent developer, but it's probably necessary.

- It is definitely not true that as a big university we don't care about things like the number of clicks it takes to do stuff. Specifically with regard to site administration, that is a main driver for some of our customisations or local plugins. For example even in the nicer moodle 2 interface, applying a set of role overrides to an activity still takes a lot of clicks and has potential for error, so we will be building a local plugin that lets our administrators apply 'canned permissions' (a whole set of overrides at once to an activity, based on templates).

We have about 10 staff responsible for maintaining all the complicated bits (ie permissions, etc - not moderating forums, but the detailed parts of site setup and responding to problems reported by the course academics, tutors etc) for something like 500 live course sites at any one time, for something like 150,000 students at any one time. I'm sure those people would say we don't really meet the objective of keeping their workload under control, but it's certainly not true that we don't care about it.

--sam

Average of ratings: Useful (1)

Re: Moving from CVS to git this week!

by Oleg Sychev - Saturday, 18 December 2010, 11:58 PM

Hi, Sam. I agree with you on more points, than you think - almost on all. Just some details and differences more:

1. I have no problems at all with code quality issues. If I'm told "you code quality is insufficient, it should be improved this way" I am either improve code (usual), or, if I totally can't do this, step back. Actually these events often are quite pleasant ones, since I could learn something new in my programming skills from them (many thanks to Tim personally regarding this, working with him was a honor). Of course, I am not a match to Tim Hunt (or some other Moodle core devs) when it comes to the code(or design) quality - but I've seen much worse in the Moodle core. And there were cases where I pick an idea from core developer and take care of debugging and implementing it. I agree with that barrier and going to take it if I can do this at all. After all I just personally hate to do low-quality things. All I need is a hint how this could be better. I also find out, that in practice it's not unadequate code quality that stalled code going to the Moodle. Developers usually could relatively fast tell that code is of low quality and write about it. It's when the code is finally of good quality that thing is usually stalled, since developer now feeling much more responsible and want to be very sure it won't hurt anything, which is much harder to find time and concentration for.

2. I also respect Moodle core devs right to make decisions, who thing should be implemented - that's their project after all. There were cases where I would prefer another way, but do as I was told by core devs. Or even one, where I agreed to do things as local mod, but write a refactoring to concentrate code in one place, which costs moodle about a hundred of strings of code. No problem with advance agreement too.

3. And there is just this problem: "this is a good thing to have, but I'm too busy so wait and write more good code for Moodle in the meantime (sic!)". This is the barrier I can't take, the only really hard one. And I could first-hand assure you, that it is not at all easier thing to do when you have good code for Moodle quiz/questions that isn't lined up with OU objective, especially last year. You could just look on MDL-20251 (how long should we have that dumb interface that moves category only one step away for one page reload?!) or MDL-6340 to see that those words are not empty. All agree this is useful thing, reported code quality problems are fixed - and we almost could celebrate year's anniversary of this code rotting on the tracker.

4. There are two differences more between large University-initiated installation-development and teacher-initiated one:

4.1. You have a staff working directly for moodle development, who could maintain you patches all the way until they are accepted. I have none, my university didn't get me specific time for this development (unless I use it as a task in a course I teach). All work is done in my free time. I am just a user - who happens to have programming skill - who are willing to spend some take to make system he works with better for himself and others. So the problem of upgrading my Moodle core code while it is waiting is far more important for me, as I have no special time for that. I'm already looking uneasy about upgrade all this for 2.0, but having this for 2.1 etc is a real shudder. Sometime I would just give up, since it would be easier to work with dumb interface than support it's improvments personally in free time.

4.2. You example about site administration is also a good one. University-initiated development look for admin site first. I am admin and a teacher, but as so I know that teacher work done much more times than administring, so I am concerned more with question editing issues or bulk activities editing ("You are going to use new computer lab this year! - Great, now I have 30+ quizzes to manualy edit IP filtering!!!") for example.

5. I am too sceptical about "more receptive", until, at least, there will be some visual stats for Martin & others to look on how good - or bad things going in this area. And to anyone read this long, a real proposed thing: Helen (supposedly Community Manager), Anthony (supposedly Contrib Coordinator), I, Davo and any other interested core/external devs discussing and planning how this stats could be done, what tracker enchances would be needed. A separate project could be good, since then we could have special states and resolutions (like "bad code quality" etc), which would make stats much more informative. Or at least some custom fields like "has external patch", "last external patch submission date" or "core dev opinion of the patch (list)" etc. We coudn't get things moving without a picture of what's happening and how it evolves.

P.S. Moodle main development problem is how good (compared to competitors) it is. It just doesn't have enought reasons to try hard to be better for it's users. They have no choice anyway for now.

Average of ratings: -

Re: Moving from CVS to git this week!

by Tim Hunt - Sunday, 19 December 2010, 6:42 AM

Well, I have definitely not had enough time to do code review, or engage sufficiently with the work of being Moodle quiz maintainer for at least the last year. I do think that the new question code I am working on is good for Moodle in the long term, and hopefully it won't be too much longer before that is in Moodle 2.1 development version - at which point the OU will not be running radically different quiz code from standard Moodle, and my job will be a lot simpler.

I do think that when the new systems/processes have settled down, it will be much easier to see which patches are waiting for review by who. Also, I think that the tools git makes available will make code-review a lot easier. Code review is still fundamentally about people and time, but nice tools make it easier and more efficient to review things. (I am thinking particularly of github pull-requests, and its facilities to allow in-line commenting on patches.)

Also, one of the whole points of the new processes is that Martin realises that there are too many old patches languishing in the tracker. And his solution to this is to tell three of the best staff he is paying that they should spend one day each week reviewing other people's code, and helping them make it better. That is significant. Of course, at the same time, the processes are being changed, so that more review is necessary, so that is not a straight gain of code-review time. But this is a significant attempt to make things better, so let us at least give it a chance to work.

P.S. Petr just rejected one of the patches I sent for review last week

Average of ratings: -

Re: Moving from CVS to git this week!

by Petr Skoda - Monday, 20 December 2010, 1:21 AM

Hehe, yes I did. I am sure you will resubmit it for the next week and it will be accepted without any comments. I hope my own patches will get detailed review and will be rejected too if there are any potential problems found.

Thanks everybody for the patience and support. I hope sincerely that the current changes will in the longer term lead to higher code quality and improvements in our communication.

Petr

Average of ratings: -

Re: Moving from CVS to git this week!

by Ralf Krause - Thursday, 16 December 2010, 11:29 PM

Hi all,

Moodle4Mac uses cvs updates but it does not work anymore.
What do I need to do to get the update back via cvs or git.

Best regards, Ralf

Average of ratings: -

Re: Moving from CVS to git this week!

by Tim Hunt - Friday, 17 December 2010, 1:02 AM

"it does not work" is not a sufficient error report if you want help.

What exactly does it do?

Average of ratings: -

Re: Moving from CVS to git this week!

by Ralf Krause - Saturday, 18 December 2010, 10:31 PM

"it does not work", no it doesn't.

My script is shown on the docs page http://docs.moodle.org/en/Complete_Install_Packages_for_Mac_OS_X#How_To_Update_Your_Moodle4Mac

The script does not a newer version than the one from 20101214. I want to get the current version of the day but the script does not work. My script asks for the HEAD version which should get all files until today.

Best regards, Ralf

Average of ratings: -

Re: Moving from CVS to git this week!

by Tim Hunt - Sunday, 19 December 2010, 4:06 AM

Ah, I see now. This is a temporary interruption while the switch from CVS to git.

Once everything is running smoothly, CVS will be updated regularly to match the latest version in git, and then the Moodel4Mac update will work again.

Until then, you will have to wait. Hopefully not long.

Average of ratings: -

Re: Moving from CVS to git this week!

by Ralf Krause - Sunday, 19 December 2010, 5:33 AM

Could it be better to update Moodle4Mac with git?
How should I do this? Would it be easy to tell more about git?

Ralf

Average of ratings: -