I want to do this...
* git.moodle.org
** my organisation's git repository (pulled from git.moodle.org)
*** developer 1 (local git)
*** developer 3 (local git)
*** etc.
So, probably nothing unusual. I want to collaboratively develop Moodle stuff pushing and pulling from my local upstream repository.
On top of that I want the upstream repository itself to pull updates from the upstream git.moodle.org. What's the best way to organise this and/or what are the pitfalls.
Part II
Some of the developers are already working with git and pulling directly from git.moodle.org. How can I put their current work into new branches (I assume I mean branches) in the local repository and not break it horribly?
Ta
Re: Local collaboration and Git - Yeah another git question!!
Re: Local collaboration and Git - Yeah another git question!!
Re: Local collaboration and Git - Yeah another git question!!
I think I'm getting there now...
Firstly, do not worry about developers pulling updates direct from git.moodle.org, git is very clever at spotting changes that have already been applied and doesn't try to add them again.
If you want each developer to have a branch in a shared repository, you can (as far as I have been able to work out from personal experimentation) do the following.
1. set up your shared repository with: git init --bare --shared OR git clone --bare --shared git.moodle.org (the latter is probably what you want)
(I'm going to assume, from now on, that the full path to this repository is 'mymoodle.git')
2. get each developer to push to their own branch on this repository:
git push mymoodle.git master:developername
(that takes their master branch and copies it to the shared repository in a (new) branch called 'developername', with each developer putting their own branch name in after the colon)
(note 2 - they could also do: git push mymoodle.git someotherbranch:developername instead)
3. To make things more convenient for the developers, they probably want to do the following config settings:
git config remote.origin.url mymoodle.git
git config remote.origin.fetch +refs/heads/*:refs/remotes/origin/*
git config branch.master.remote origin
git config branch.master.merge developername
Now if they do a 'git pull' or 'git push' from their 'master' branch, it will be sent to/from their personal branch in the shared repository
4. If someone wants to grab the work done by another person, they could type:
git pull origin/name_of_other_developers_branch
and that would pull all the changes from that branch into their current branch
OR, if they just wanted to have a look at another branch, then a good method would be the following:
git config --global branch.autosetupmerge true (This line may no longer be necessary, I think this is now the default)
git checkout -b name_of_other_developers_branch origin/name_of_other_developers_branch
This will make a local copy of the other developer's branch that will be automatically updated whenever any 'git pull' operation is done (although a 'git merge' will also be needed if that branch is not currently checked out).
This may be the wrong way to use git, and I am willing to stand corrected, but I think it *should* work.
Thinking about it, you could also do just step 4 above, to pull another developers work into a branch on YOUR development space (rather than going via a shared repository), with the command being something like this:
git remote add sensible_alias_name path_to_other_developers_work
git checkout -b local_name_for_branch sensible_alias_name/branch_you_want
I hope that's not too confused...
Re: Local collaboration and Git - Yeah another git question!!
I think it's more likely that the branches on the shared respository would be different projects (that multiple devs would work on). I'm reasonably confident about that though.
I guess then that someone comes along every so often and does 'git pull' on the shared repository to get the latest stuff from git.moodle.org. I'm much less sure how that might be merged into project branches if that is required. Can that just be done in the same was as you do it locally?
I don't think I've seen that 'colon' notation before. What does that do?
Re: Local collaboration and Git - Yeah another git question!!
e.g. 'git push origin master:davomaster' would create a new branch on the remote repository pointed to by 'origin', the branch would be called 'davomaster' on the repository and would be linked to the 'master' branch in my local development repository.
As for merging in the latest changes, the safest way would be for someone who is working on that branch to pull from 'git.moodle.org' (as that way, they can deal with any conflicts / breakages), but you may be able to get away with going to the shared repository and then calling:
git pull git.modle.org (which will pull the latest updates into master)
git push . master:name_of_other_branch (which will push the updates from the master branch into the other branch, but will only work if there are no merge conflicts, the other developers would then need to do a 'git pull', just the same as when any other changes had been pushed to their branch on the shared repository)
Remember, that if you are on a branch that already has updates pulled in from 'git.moodle.org' and then you pull in changes from another branch that has not had the updates from 'git.moodle.org', then this will either work without problems (90% of the time) or will require exactly the same merge conflict resolutions as would have been needed if the branch *had* pulled from 'git.moodle.org' (at the time when that pull was done, if you get my meaning).
Similarly, as already mentioned, it is perfectly safe for two people to independently update from 'git.moodle.org' and then merge their changes together.
Re: Local collaboration and Git - Yeah another git question!!
I'm not sure I fully understand push and pull yet
Re: Local collaboration and Git - Yeah another git question!!
Ooohh.... my head's hurting again.... I'm not really clear on the best way of getting Moodle updates. So if you pull (locally) and then do a push to the shared repository, this will deal with the all (moodle) branches or just the bit you're working on.
If you don't specify any branch names in your push commands, it depends on how you have configured your remote repository (ranging from only a small subset to all the branches you fetched/cloned from there). So the easier way to do it is always specify the branches you want to push:
git push shared-repo my-local-branch-to-push my-other-local-branch-to-push ...
Saludos. Iñaki.
Re: Local collaboration and Git - Yeah another git question!!
but you may be able to get away with going to the shared repository and then calling: git pull git.modle.org (which will pull the latest updates into master)
This won't work, because you can't pull on a 'bare' repository (there's no working copy to merge the changes with). You need to 'fetch' instead of 'pull', but that won't merge the changes into your local branches unless you do that manually, so that's why I thinkg pulling from git.moodle.org on developers local repos and then pushing to the shared repository is the way to go.
Saludos. Iñaki.
Re: Local collaboration and Git - Yeah another git question!!
I want to do this...
If you can read Spanish , I've got a detailed document that I submitted to the Spanish Barcelnoa '08 Moot which talks specifically about this. It uses Catalyst's repo in the examples (git.moodle.org wasn't available at that time), but it's otherwise still 100% valid.
You can get it at http://tinyurl.com/4eyj3s. It's under a CC-BY-SA license.
On top of that I want the upstream repository itself to pull updates from the upstream git.moodle.org. What's the best way to organise this and/or what are the pitfalls.
Some of the developers are already working with git and pulling directly from git.moodle.org. How can I put their current work into new branches (I assume I mean branches) in the local repository and not break it horribly?
You add a new remote repository pointing to each developer's git repo specifying the remote branches you want to fetch. Say you want to fetch my 'MOODLE_19_STABLE_LDAP_REFACTOR' branch (as a tracking branch) from my ldap-refactor.git repository at git.mondragon.edu (calling it 'mondragon'), and fetch the branch inmediatly:
git remote add -t MOODLE_19_STABLE_LDAP_REFACTOR -f mondragon git://git.mondragon.edu/ldap-refactor.git/
Now you can create a local branch (I prefer to create them as tracking branches, but this is optional) based on that remote branch with:
git branch --track my-19-ldap-refactor mondragon/MOODLE_19_STABLE_LDAP_REFACTOR
and hack it to pieces without breaking anything at all, except your local branch
Saludos. Iñaki.
Re: Local collaboration and Git - Yeah another git question!!
You've lost me, sorry.....
I can't see what 'git remote add' does. I read the man page and that's made the confusion worse
Are you sure we're talking about the same thing? Say I've been working on my local machine with git and I've cloned git.moodle.org and created some branches and done some work. Now another developer has been doing the same thing and want to work together. How do I make that work. I think we need to integrate our current branches into a common repository on our dev server but I don't see how.
Re: Local collaboration and Git - Yeah another git question!!
I can't see what 'git remote add' does.
'git clone' does several operations on your behalf:
- It creates an empty local repository.
- It adds a remote repository, called 'origin' by default.
- It fetches the objects (files, directories, commits, etc.) and references[1] (branches, tags, etc.) from the origin repository. By default it fetches all the objects and references of the remote repository.
- It creates a local branch called 'master' from the remote head 'HEAD' branch
- It checks out the master branch to the working directory.
Plus a few other details that are not relevant in this discussion.
[1] In git a reference is a 'pointer' to a commit object (to any commit object in the whole repository history), and a head is a reference for the last commit of a given commit tree. A branch is a head and all the commits leading to that head (i.e., the head history).
If you want to work with branches from different remote repositories in your local repository, you can't use 'git clone' for this (as it creates a new local repository in step 1., which you don't want). So you need to perform some or all of the steps yourself.
That's where 'git remote' comes into play. 'git remote' allows you to perform steps 2 and 3 and select wich references you want to fetch in step 3, and then 'git branch'/'git checkout' allow you to perform steps 4 and 5.
Then you can work on your local branches. When you want to push your local branches to a remote repository, you first need to tell git what to push there (which local branches) and where to (into what remote branches). And you use 'git config' for that.
I know this is going to be a bit longwinded (it already is ), but I'll show all the commands to setup the whole thing and the regular workflow (I'll be rather terse, the full details are on the paper mentionned before .
Some names:
- devel-1: the first developer
- devel-2: the second developer
- shared-repo: the name of the shared repository directory
Some assuptions:
- Linux/Unix environment
- All the repositories are on the same machine (if not, only need to change git transports from git:// to git+ssh://, etc.)
Devel-1 local repo:
Create & config:
cd /path/to/devel-1/dir
mkdir devel-1.git
cd devel-1.git
git init
git config --global user.name 'Devel-1 Real Name'
git config --global user.email 'devel-1@email.address'
git config --global i18n.commitEncoding 'utf8'
git config --global i18n.logOutputEncoding 'utf8'
Get branches from git.moodle.org repo:
git remote add -t cvshead -t MOODLE_19_STABLE -t MOODLE_18_STABLE -m cvshead moodleorg git://git.moodle.org/moodle.git
git fetch moodleorg
Create local (work) branches:
git branch --track cvshead-devel-1 moodleorg/cvshead
git branch --track mdl19-devel-1 moodleorg/MOODLE_19_STABLE
git branch --track mdl18-devel-1 moodleorg/MOODLE_18_STABLE
Shared repo:
Create & config:
cd /path/to/shared-repo/dir
mkdir shared-repo.git
cd shared-repo.git
git --bare init --shared=all
chmod g=rwxs,o=rx . # Give appropiate permissions to users in
sudo chgrp -R git-moodle . # group 'git-moodle'.
Populate shared repo from local devel-1 repo:
cd /path/to/devel-1/dir/devel-1.git
git remote add shared-repo /path/to/shared-repo/dir/shared-repo.git
git push shared-repo +cvshead-devel-1: +mdl19-devel-1: +mdl18-devel-1:
Devel-1 local repo:
Configure local branches to merge the right remote branch when pulling:
We need to do this only for the first developer, as the shared repo didn't exist when we created the branches.
cd /path/to/devel-1/dir/devel-1.git
git config branch.cvshead-devel-1.remote shared-repo
git config branch.cvshead-devel-1.merge refs/heads/cvshead-devel-1
git config branch.mdl19-devel-1.remote shared-repo
git config branch.mdl19-devel-1.merge refs/heads/mdl19-devel-1
git config branch.mdl18-devel-1.remote shared-repo
git config branch.mdl18-devel-1.merge refs/heads/mdl18-devel-1
Devel-2 local repo:
Create & config:
cd /path/to/devel-2/dir
git clone -o shared-repo /path/to/shared-repo/dir/shared-repo.git devel-2.git
cd devel-2.git
git config --global user.name 'Devel-2 Real Name'
git config --global user.email 'devel-2@email.address'
git config --global i18n.commitEncoding 'utf8'
git config --global i18n.logOutputEncoding 'utf8'
Get branches from git.moodle.org repo:
cd devel-2.git
git remote add -t cvshead -t MOODLE_19_STABLE -t MOODLE_18_STABLE -m cvshead moodleorg git://git.moodle.org/moodle.git
git fetch moodleorg
Create local (work) branches:
From moodle.org:
git branch --track cvshead-devel-2 moodleorg/cvshead
git branch --track mdl19-devel-2 moodleorg/MOODLE_19_STABLE
git branch --track mdl18-devel-2 moodleorg/MOODLE_18_STABLE
From the shared-repo:
git branch --track cvshead-devel-1 shared-repo/cvshead-devel-1
git branch --track mdl19-devel-1 shared-repo/mdl19-devel-1
git branch --track mdl18-devel-1 shared-repo/mdl18-devel-1
Configure local branches to merge the right remote branch when pulling:
Nothing to do.
Now you can pull and push from any of the configured remote repositories (git.moodle.org and your shared local repo) by specifiying the remote repository name when using 'git pull' or 'git push'
Clear as mud, isn't it
If I can find a couple of hours, I'll dump all this into the Moodle Docs and add some useful comments around the commands.
Saludos. Iñaki.
Re: Local collaboration and Git - Yeah another git question!!
I'm going to read this carefully when I'm more awake
Expect more questions
Re: Local collaboration and Git - Yeah another git question!!
My immediate reaction is that it seems spectacularly complicated.
Why can I not just clone git.moodle.org as a bare repository to be shared repository and then my devs clone from that to their personal repo? That's intuitively what I would have tried.
I would have expected to have to go to the shared repo and do a 'pull' every morning but I can live with that.
Nothing's ever as simple as it first seems with git
Re: Local collaboration and Git - Yeah another git question!!
My immediate reaction is that it seems spectacularly complicated. Up to a certain point, I agree with you. But then you need to remember that you are trying to use a distributed tool in a centralized mode. While possible, it's a bit more convoluted because it was designed with a different work flow in mind.
Why can I not just clone git.moodle.org as a bare repository to be shared repository and then my devs clone from that to their personal repo? That's intuitively what I would have tried.
The problem with this scenario is that while objects (files, directories, commits, etc.) are global, references (heads, branches, tags, etc.) are local. In other words, the MOODLE_19_STABLE branch in git.moodle.org and the MOODLE_19_STABLE branch in mi local repository have the same commit history (up to a certain point), but they are two different references.
This means that making some more commits in git.moodle.org/MOODLE_19_STABLE doesn't make local-repo/MOODLE_19_STABLE pick up those changes. If I do a 'fetch', I get the objects of those commits locally, but I don't touch the local reference. I need to do a 'merge' for that to happen (btw this is what 'pull' does: a fetch followed by a merge).
But a merge needs a working copy (i.e., a checked out copy of the reference) to apply the changes and record the new state in a new commit. So you while you can do this in a regular repo, you can't do it in a bare repository. That's why I propose to do all the merges/pulls in the developer repositories and then push the results (if needed) to the shared repository.
By the way, if you are thinking about using a non-bare repository on the shared repo to avoid this problem, don't do it . It's a very bad idea. Pushing to non-base repositories can have all sort of problems if the checked out branch in the shared repo and the branch you are pushing are not the same. I seem to remember git develpers were thinking about checking for this and aborting the push operation altogether.
I would have expected to have to go to the shared repo and do a 'pull' every morning but I can live with that.
You'd need to pull twice anyway: once in your shared repo from git.moodle.org, and a second time in each developer repo from the shared repo. You can save some bandwith (and a little bit of time if your WAN is much slower than your LAN), but that's pretty much the only difference I can think of.
Saludos. Iñaki.
Re: Local collaboration and Git - Yeah another git question!!
Howard -- tracking a 'local' project with various developers in a simple way does require having a local repository (there is actually an alternative strategy, but it's conceptually a bit harder).
I've done this is various SCMs and git is actually the most straightforward one.
Having said that, Iñaki is giving you the full commands that you need to use to set things up, which you do _only once_. And he's using all the key options fully spelt out. Good to get your head around it -- he's not relying on any defaults of dark magic.
Once you're setup and handy with it, it takes a single command to create a new branch on the server.
And I think I could get the setup to be done in 1/3 of the steps, but it'd be rather obscure
Re: Local collaboration and Git - Yeah another git question!!
I guess a better question might be... how do we all use git and not loose stuff?
One advantage with a central repository of some sort is that you know where the thing to back up is and you know where the first place to look is.
Re: Local collaboration and Git - Yeah another git question!!
I guess a better question might be... how do we all use git and not loose stuff?
First thing I would say: git reflog. I didn't learn about the reflog for quite a while and its a really nice way to recover from accidents. It allows me to sleep easier at night when using git commands. Makes it feel less 'fragile'. This writeup on gitready is probably better than I can do now.
But also if you think of git push as the same operation (in terms of 'backup') like svn ci and push to a remote branch on your central repository to backup then its essentially the same.
Re: Local collaboration and Git - Yeah another git question!!
[...] this should be "upgraded" to a wikipage.
"Upgraded" to http://docs.moodle.org/en/Development:Git_tips#Local_collaboration
Saludos. Iñaki.
Re: Local collaboration and Git - Yeah another git question!!
Here's what we now do at Catalyst:
- We cloned git.moodle.org into a bare repository on our office server (git.catalyst.net.nz).
- We have a cron job which fetches the latest changes every 30 minutes from git.moodle.org and updates the local git.catalyst.net.nz repo.
- Developers pull from the Catalyst git repo and push their own feature branches there as well.
- Each developer is responsible for merging the local copy of MOODLE_xx_STABLE into their own feature branch.
Note that in a setup like this, it's of course important that nobody pushes directly to the local copy of the MOODLE_xx_STABLE branches.
Cheers,
Francois