Automate CVS update and integrate with our own SVN server question

Automate CVS update and integrate with our own SVN server question

Wen Hao Chuang -
回帖数:11
Dear all:

I'm sure a lot of you are facing this issue every day. Just want to see what is your current solution and if you have more "details" about your set up and could share with us, that would be great.

Currently for our development environment we have a CVS branch that is following Moodle's nightly build. However, we have to merge the daily CVS update with our own customization manually, and we would be very interested to know more how people are setting up in the way that new fixes from 1.8 core can be pushed into our SVN automatically (is there any scripts that you guys are using?) We also want to know more about possibly setup that changes from core won't touch anything of our local customization changes and if so happen it would be automatically flagged for review. Have anyone setup such a dev environment already and if so could you please share with us with all of the details? Many thanks!
回复Wen Hao Chuang

Re: Automate CVS update and integrate with our own SVN server question

Martín Langhoff -
I very strongly recommend that you use GIT (rather than SVN) to track your local customisations. I've misspent way too many days and nights scripting my way around the limitations of CVS, SVN, Arch... and they are not suited to the task.

Search this forum for earlier discussions 眨眼
回复Martín Langhoff

Re: Automate CVS update and integrate with our own SVN server question

Samuli Karevaara -
Do cvs2git or git-fast-import require more than a read-only access to the cvs repo? (I couldn't find info on them with Google... cvs2svn requires a filesystem level access to the cvs repo 伤心 )
回复Samuli Karevaara

Re: Automate CVS update and integrate with our own SVN server question

Iñaki Arenaza -
Core developers的头像 Documentation writers的头像 Peer reviewers的头像 Plugin developers的头像
git-cvsimport can work with either a remote repository (with read-only access) or a local copy of the repository (I mean the real repository, not a working copy).

By the way, you can get a copy of the repository via rsync from Sourceforge.net if you want (that's what the new anonymous CVS mirrors are already doing). Just follow the instructions given here:

http://docs.moodle.org/en/How_to_set_up_a_CVS_mirror#Mirroring_Moodle_repository

Saludos. Iñaki.
回复Iñaki Arenaza

Re: Automate CVS update and integrate with our own SVN server question

Samuli Karevaara -
I should have known! Thanks! (After all, that's part of what the "open" in open source means 微笑 )
回复Samuli Karevaara

Re: Automate CVS update and integrate with our own SVN server question

Martín Langhoff -
As Iñaki points out, it works over pserver (that is, talking to a remote repo) but it's WAY faster if you rsync the repo to the local disk and run git-cvsimport from there.

However, you don't need to do that -- we do already. There's a CVS-to-GIT gateway that we run, and gets a daily import from CVS. Just do

git clone http://git.catalyst.net.nz/git/moodle-r2.git

NOTE: There's been some messing about with MOODLE_16_STABLE in CVS repo, and that has confused GIT quite a bit. It's in my list to fix it up, but _that particular branch_ isn't 100% correct on the git repo.
回复Martín Langhoff

Re: Automate CVS update and integrate with our own SVN server question

Wen Hao Chuang -
Hi Martin L (and others):

Just curious, have anyone tried out Bazaar version control software (http://bazaar-vcs.org/)? How does this compare to GIT? I wonder if bazaar provide any vendor branch guildlines at all? Thanks!

Also might want to read this article ("Choose lossless VCS tools if you have that luxury") at http://www.markshuttleworth.com
回复Wen Hao Chuang

Re: Automate CVS update and integrate with our own SVN server question

Martín Langhoff -
Yes. Martin Pool was as frustrated as I was with Arch when he started coding bzr and then bzr-ng which is now "bazaar". I was following his blog and joined some architectural discussions in gnuarch mailing list.

Over time, bzg-ng went from having an Arch/TLA-likle approach to having a git-like approach (once git architecture was defined, lots of people decided to copy it 眨眼. It does have some "vendor branch tracking" facilities. As far as I've seen lately, it covers a subset of what git can do, and it's quite a bit slower. Initially it had the claim of being more portable, but Mercurial is _much_ better as a DSCM written in a portable scripting language. And git now runs on windows _really_ well (google for GIT MinGW).

For a fun version of the above discussion, see http://weblogs.mozillazine.org/preed/2007/04/version_control_system_shootou_1.html
but bear in mind that it's outdated - git does Windows too 眨眼

And there's a little anecdote around GIT vs Bazaar 眨眼 I met MartinP January last year at LinuxConfAu, where he was doing a presentation on Bazaar (Mark was there too) and I was doing one on GIT (Linus turned up - good way to scare your presenter!).

Jim Gettys turned up at the Bazaar presentation and asked if it was mature enough to port a large project he had been working on for about 20 years. MartinP said "not just yet". Later that day I asked Jim if perhaps I could help him try with GIT and shake out any bugs we found.

So later we sat down with Jim, Keith Packard and Carl Worth and started exploring how could we migrate that huge project, 20-odd years worth of changes on a huge codebase, into GIT. We didn't finish it there, but you can see the results here
http://wiki.x.org/wiki/ModularDevelopersGuide#head-a085b56a4b98f5418566ff5f575ca47805bd54c6

And here's the web view
http://gitweb.freedesktop.org/

Early this year I tried help the Mozilla project, but Windows support wasn't there yet, and I was quite busy. It didn't quite work out.
回复Wen Hao Chuang

Re: Automate CVS update and integrate with our own SVN server question

Martín Langhoff -

I hadn't read the article... Mark is a very smart guy, and he is trying to make the case for Bazaar, which is slower, less portable and with less features than GIT or Mercurial.

Hmmmm.

Rename tracking is not explicit in GIT, and that is a feature. GIT is so fast that it autodetects renames -- it also autodetects file splits and file joins, and functions or chunks of code moving from one file to the other. This is specially visible when you do git-blame (which is annotate).

The SCMs that track "file renames" do so by requiring you to record it explicitly. And Linus said many times that he's not interested in tracking renames explicitly because they are only one case of moving content, and not the most interesting case either. The really interesting cases are things that happen during refactoring and modularisation: several copies of the same code consolidate, files are split, etc. And you can't count on people recording that explicitly.

So - while not recording renames, git will youw you renames, copies and a few other things. If you've started mod/bla by copying mod/forum and making a few changes, git blame will show you the real ancestry of that bit of code. BTW, the newfangled "gui blame" rocks.

The other point he touches is speed. And I agree - at first blush for small/medium projects, speed isn't that important. Except that there are some problems with that idea...

First - by being incredibly fast in its core operations, you can then afford to do a lot more. For example - doing simple merges in git is very fast, and internal lookups of file versions are instant, so during the merge phase git does a lot of "trying to see if this works better" 3-way merges on a file-by-file basis, and there are plans for it to get smarter. That "let's be smart" mode would take long minutes with anything else.

Second -- once you are working with one large project... you have to use the fast tool. And then you can use it in the small projects too, no penalty! Why would you use 2 tools? So which project gets more developers, more attention, more polish?

And third - Moodle is not "small/medium" by Bazaar's definition: my current Moodle HEAD checkout has 4290 files. It's much smaller since the lang splitoff: the last release before that, v1.5, has 12K files.

At the moment, it's Mercurial vs GIT. Bazaar as a project had a strange timing... it started at the right time, but took ages to deliver anything usable. And then it kept morphing a lot. Around the same time, Darcs was a lot more advanced, and had a better UI. Mercurial and GIT were built in a fraction of the time Bazaar has been in development, both are faster and more featureful, and have a stronger dynamic that you can see in their mailing lists.

And if you look at the code and internal architecture, Mercurial and GIT have a lot more clarity of thought. At least in my reading. Matt Mackall was at LinuxConfAu, and he's a damn smart guy, and keeps the fire going in the Mercurial project.

ANYWAY - this has been a good long rant. Bazaar is part of the new DSCM generation. It's a good tool -- you won't suffer serious pain doing vendor branch tracking with it, like you would with SVN or CVS. IMHO it's not as good as GIT or Mercurial by a long mile, so my take is that you'll save a bit of time going straight to GIT. 微笑

And Darcs, Bazaar, Mercurial and GIT are all based on very similar principles, their commands and options are similar, and they often feed off each other - gitk was cloned in bzrk for example. So you can migrate across them easily.

回复Wen Hao Chuang

Re: Automate CVS update and integrate with our own SVN server question

Dan Poltawski -
Hi Wen,

I've looked at using Bazar, mostly just because it looks like a smaller learning curve than GIT and I just wanted a simple DSCM for tracking my changes ;) (But also we're looking at using it at work more).

Once nice side-effect of the Canonical ties with bazaar is that you can get Canonical to do the upstream tracking for you with launchpad.net ( https://launchpad.net/moodle/trunk ). ;)

Unfortunately the bazaar import is failing - I had a very helpful conversation with the one of the launchpad import admins a few weeks ago trying to get to the bottom of this problem. It turns out to be a stray file named \ in the CVS repo. I've opened MDLSITE-199 to see if we can look at removing this from the CVS repository as a quick (selfish) win[1]. But also there is a bazaar bug open for it here.

Anyway - long story short - I haven't actually used bazaar yet so have no useful information to add to the DSCM debate. :D

[1] but as I say in the bug, there may be other systems which have problems with that filename - and I would argue there isn't much to be lost by loosing that one bit of history ;)
回复Wen Hao Chuang

Re: Automate CVS update and integrate with our own SVN server question

Bryce Thornton -
http://moodle.org/mod/forum/discuss.php?d=34472

This is a link to a similar question I had a while back. We still use SVN and follow their vendor branch guidelines to merge changes in. It's not perfect, but it works for now. Here's documentation on how to maintain a vendor branch in SVN.

http://svnbook.red-bean.com/nightly/en/svn.advanced.vendorbr.html
回复Wen Hao Chuang

Re: Automate CVS update and integrate with our own SVN server question

Jeff Graham -
Hi Wen,

I have done the following before and it seems to work quite well for lightweight vendor branch tracking. Please note that this may have problems for more complex situations; I haven't experienced any. If you do you would probably do best to heed Martin's advice assuming you have the time to invest.

Do a moodle CVS checkout. Add this checkout with CVS tags to your SVN repo.

Now here is the workflow;
Ensure your svn checkout is in synch. 'svn status' so that you are only committing CVS update changes.

Sync with CVS.

cvs -z3 update -dP

Review the messages;
  • resolve any conflicts
  • add new files
  • remove deleted files
Commit CVS changes to your SVN repo: Use svn status to help catch any files you may have not queued to delete or add prior to your svn commit.

Now you can get your local changes with a cvs diff. Ignore any files you have added locally or the bunch of lines at the top ending in '.svn'

From my experience this has worked quite well and doesn't require much extra time to learn or setup if you are already familiar with SVN and CVS.

regards,
Jeff