Hi Guys,
I track the code for our moodle install using Git, with the repo cloned from git.moodle.org. Every monday run git fetch to get the latest bugfixes from the MOODLE_19_STABLE branch, which worked fine up until the CVS moved to the new server. Now, whenever I fetch an update and try to merge it into my local code, I get a HUGE number of conflicts and whitespace issues which take me ages to go through and fix. Most of the conflicts seem to be caused where ereg_replace is being replaced with preg_replace, most confusingly in areas I've not even touched before with local modifications.
Has anyone else hit this problem? Am I doing something wrong?
I merged the latest 19_STABLE from git.moodle.org yesterday and hit no problems at all.
I suppose it would take an example of one of the conflicts to comment further.
I suppose it would take an example of one of the conflicts to comment further.
Here's a few of the conflicts:
admin/mnet/MethodTable.php:467
<<<<<<< HEAD:admin/mnet/MethodTable.php
$wordBase0 = preg_replace('/^[&$]+/','',$word0);
$wordBase1 = preg_replace('/^[&$]+/','',$word1);
$startBase = strtolower(preg_replace('/^[&$]+/','',$start));
=======
$wordBase0 = ereg_replace('^[&$]+','',$word0);
$wordBase1 = ereg_replace('^[&$]+','',$word1);
$startBase = strtolower(ereg_replace('^[&$]+','',$start));
>>>>>>> origin/MOODLE_19_STABLE:admin/mnet/MethodTable.php
auth/shibboleth/logout.php:142
<<<<<<< HEAD:auth/shibboleth/logout.php
$session_key = preg_replace('/sess_/', '', $file);
=======
$session_key = ereg_replace('sess_', '', $file);
>>>>>>> origin/MOODLE_19_STABLE:auth/shibboleth/logout.php
lib/adodb/drivers/adodb-sybase.inc.php:368
<<<<<<< HEAD:lib/adodb/drivers/adodb-sybase.inc.php
function ADORecordSet_array_sybase($id=-1)
{
$this->ADORecordSet_array($id);
}
// sybase/mssql uses a default date like Dec 30 2000 12:00AM
function UnixDate($v)
{
global $ADODB_sybase_mths;
//Dec 30 2000 12:00AM
if (!preg_match( "#([A-Za-z]{3})[-/\. ]+([0-9]{1,2})[-/\. ]+([0-9]{4})#"
,$v, $rr)) return parent::UnixDate($v);
if ($rr[3] <= TIMESTAMP_FIRST_YEAR) return 0;
$themth = substr(strtoupper($rr[1]),0,3);
$themth = $ADODB_sybase_mths[$themth];
if ($themth <= 0) return false;
// h-m-s-MM-DD-YY
return mktime(0,0,0,$themth,$rr[2],$rr[3]);
}
function UnixTimeStamp($v)
{
global $ADODB_sybase_mths;
//11.02.2001 Toni Tunkkari toni.tunkkari@finebyte.com
//Changed [0-9] to [0-9 ] in day conversion
if (!preg_match( "#([A-Za-z]{3})[-/\. ]([0-9 ]{1,2})[-/\. ]([0-9]{4}) +([0-9]{1,2}):([0-9]{1,2}) *([apAP]{0,1})#"
,$v, $rr)) return parent::UnixTimeStamp($v);
if ($rr[3] <= TIMESTAMP_FIRST_YEAR) return 0;
$themth = substr(strtoupper($rr[1]),0,3);
$themth = $ADODB_sybase_mths[$themth];
if ($themth <= 0) return false;
switch (strtoupper($rr[6])) {
case 'P':
if ($rr[4]<12) $rr[4] += 12;
break;
case 'A':
if ($rr[4]==12) $rr[4] = 0;
break;
default:
break;
}
// h-m-s-MM-DD-YY
return mktime($rr[4],$rr[5],0,$themth,$rr[2],$rr[3]);
}
=======
function ADORecordSet_array_sybase($id=-1)
{
$this->ADORecordSet_array($id);
}
// sybase/mssql uses a default date like Dec 30 2000 12:00AM
function UnixDate($v)
{
global $ADODB_sybase_mths;
//Dec 30 2000 12:00AM
if (!ereg( "([A-Za-z]{3})[-/\. ]+([0-9]{1,2})[-/\. ]+([0-9]{4})"
,$v, $rr)) return parent::UnixDate($v);
if ($rr[3] <= TIMESTAMP_FIRST_YEAR) return 0;
$themth = substr(strtoupper($rr[1]),0,3);
$themth = $ADODB_sybase_mths[$themth];
if ($themth <= 0) return false;
// h-m-s-MM-DD-YY
return mktime(0,0,0,$themth,$rr[2],$rr[3]);
}
function UnixTimeStamp($v)
{
global $ADODB_sybase_mths;
//11.02.2001 Toni Tunkkari toni.tunkkari@finebyte.com
//Changed [0-9] to [0-9 ] in day conversion
if (!ereg( "([A-Za-z]{3})[-/\. ]([0-9 ]{1,2})[-/\. ]([0-9]{4}) +([0-9]{1,2}):([0-9]{1,2}) *([apAP]{0,1})"
,$v, $rr)) return parent::UnixTimeStamp($v);
if ($rr[3] <= TIMESTAMP_FIRST_YEAR) return 0;
$themth = substr(strtoupper($rr[1]),0,3);
$themth = $ADODB_sybase_mths[$themth];
if ($themth <= 0) return false;
switch (strtoupper($rr[6])) {
case 'P':
if ($rr[4]<12) $rr[4] += 12;
break;
case 'A':
if ($rr[4]==12) $rr[4] = 0;
break;
default:
break;
}
// h-m-s-MM-DD-YY
return mktime($rr[4],$rr[5],0,$themth,$rr[2],$rr[3]);
}
>>>>>>> origin/MOODLE_19_STABLE:lib/adodb/drivers/adodb-sybase.inc.php
I'm about to grossly exceed my knowledge of git here and probably make a fool of myself. It's because the merge is not fast-forward so something has happened to your local branch to throw off the merge (or more precisely *cause* a full merge). Could just be something like file permission changes that have been committed causing a 'proper' merge.
One of our git experts will know
One of our git experts will know
I wouldn't expect git to attempt a fast-forward merge due to my local copy containing modifications. However, as I haven't changed the files in question locally, I wouldn't expect it to cause a conflict either. I may be wrong, though.
Git must *think* that they are changed locally surely or it would just take the newer change from git.moodle.org (as fast-forward).
Maybe I'm thinking about this the wrong way. Where's Langhoff when you need him
Maybe I'm thinking about this the wrong way. Where's Langhoff when you need him
Batsignal! Let me get into my leotard... damn, too much pasta lately!
When you are in the middle of a merge, and it shows a conflict and you wonder "but I did't change that on my side" or "what the hell are the conflicting changes anyway", you can do:
gitk --merge -- path/to/mysteriously_conflicted_file.php
and it'll show you the commits on both sides. To get an overview of the merge you're doing
gitk --merge
Without psychic powers I can't see what's the matter in Mark's repository, but the commands above should help diagnose...
Mark
you should try the commands I've suggested in my other msg. But I have a hunch here: are you mixing rebases and merges? Or perhaps applying patches from the tracker, or submitting patches to the tracker that later someone applies into CVS slightly modified?
The two cases above would explain your situation.
The case of duplicate but slightly different bugfixes, is expected; I'd review and generally prefer the CVS fix.
If you accidentally "rebased from CVS" instead of "rebase on top of the branch from CVS"... careful! Probably harmless this time, but something in your workflow is not quite right.
Git will always do the right thing when merging. Rebase always takes a bit more understanding of the situation than a merge.
you should try the commands I've suggested in my other msg. But I have a hunch here: are you mixing rebases and merges? Or perhaps applying patches from the tracker, or submitting patches to the tracker that later someone applies into CVS slightly modified?
The two cases above would explain your situation.
The case of duplicate but slightly different bugfixes, is expected; I'd review and generally prefer the CVS fix.
If you accidentally "rebased from CVS" instead of "rebase on top of the branch from CVS"... careful! Probably harmless this time, but something in your workflow is not quite right.
Git will always do the right thing when merging. Rebase always takes a bit more understanding of the situation than a merge.
Hi Martin,
I've had patches committed, but none that relate to the conflicts I'm seeing. Even so, I usually wait for the update from CVS rather than applying the patch to my main dev branch for exactly this reason
I certainly haven't rebased anytime recently (and even then, not in my main branch). gitk won't install on my server due to an odd dependency problem.
I'm going to give Penny's suggestion a go!
I've had patches committed, but none that relate to the conflicts I'm seeing. Even so, I usually wait for the update from CVS rather than applying the patch to my main dev branch for exactly this reason
I certainly haven't rebased anytime recently (and even then, not in my main branch). gitk won't install on my server due to an odd dependency problem.
I'm going to give Penny's suggestion a go!
Note that Penny's suggestion is what git does behind the scenes when you ask for a rebase.
It will help you find out what patches you have against those files (and you can decide what to do, whether to skip them, merge them...). But if you have lots of patches, and have been merging from CVS for a long time, it could be a ton of work.
The fastest way to understand WTH is going on is the gitk --merge trick I mentioned before.
It will help you find out what patches you have against those files (and you can decide what to do, whether to skip them, merge them...). But if you have lots of patches, and have been merging from CVS for a long time, it could be a ton of work.
The fastest way to understand WTH is going on is the gitk --merge trick I mentioned before.
Thanks Martín, after some persistence I managed to get gitk --merge working. However, it's not shed a lot of light yet.
It shows me the patch being applied to the conflicted files, where it's changing preg functions to ereg functions (the comment is "reverting untested egreg cleanup - not suitable for stable, sorry" so I guess it's undoing a previous commit). However, I haven't changed these lines locally, so I don't know why this is causing a conflict.
It shows me the patch being applied to the conflicted files, where it's changing preg functions to ereg functions (the comment is "reverting untested egreg cleanup - not suitable for stable, sorry" so I guess it's undoing a previous commit). However, I haven't changed these lines locally, so I don't know why this is causing a conflict.
I Just had a thought - Last time I merged MOODLE_19_STABLE I had similar issues with a few conflicts and a LOAD of whitespace problems, which I fixed with a find and replace. Could it be that I removed some whitespace (or replaced a tab) which is present in the patch it's trying to apply to the file, so It's creating a conflict?
That could explain it. Unfortunately, I don't know of git has a 'merge ignoring whitespace conflicts' 
-- will check.
Whitespace conflicts are kinda nasty, so most projects that merge lots of code back-and-forth are very tight about them (git has a "complain about trailing whitespace" mode). I think in Moodle we are a bit more laid back. Git could support this, but I'm not sure if it does.
Edit: follow answers to http://marc.info/?l=git&m=125975008109599&w=2
Whitespace conflicts are kinda nasty, so most projects that merge lots of code back-and-forth are very tight about them (git has a "complain about trailing whitespace" mode). I think in Moodle we are a bit more laid back. Git could support this, but I'm not sure if it does.
Edit: follow answers to http://marc.info/?l=git&m=125975008109599&w=2
Thanks Martín, I'll keep an eye on that thread.
So does it look like I've caused myself a problem as my local repo has trailing whitespace removed and tabs converted to spaces, but the upstream code doesn't?
Does git have a "don't complain about trailing whitespace" mode too? If so, would it be possible to use it in conjunction with Penny's suggestion to work my way out of the mess, or am I a bit stuck since all my patches will still cause whitespace conflicts where it's been removed? My brain hurts...
So does it look like I've caused myself a problem as my local repo has trailing whitespace removed and tabs converted to spaces, but the upstream code doesn't?
Does git have a "don't complain about trailing whitespace" mode too? If so, would it be possible to use it in conjunction with Penny's suggestion to work my way out of the mess, or am I a bit stuck since all my patches will still cause whitespace conflicts where it's been removed? My brain hurts...
Just as a rule, do not apply a whitespace cleanup on your branch. Such a "cleanup", or any major search/replace will only lead to pain... and a lot of it.
This is not specific to git -- just something to be careful with when you are running a customisation branch. It will be pain regardless of the project, regardless of the SCM tool you use.
What do to do? Can you perform a rebase by hand (the way Penny suggests) skipping your whitespace "fixup" commits?
The patches that come after your now skipped whitespace fixup commits may have their own whitespace conflicts. What you can do is, once git am has complained about them, do
# remove conflict markers, partially applied changes, etc
git reset --hard
# use 'patch' which does know how to ignore WS
patch -p1 --ignore-whitespace < .patches/0020-my-other-patch
# tell git about the files that have been changed
git add filea fileb
# tell git am you've done
git am --resolved
Right, that sounds like fun! I'll get stuck in when I've got a spare day...
Once I've done that, I'll have to switch off Eclipse's feature to automatically strip trailing whitespace (so I don't run into this problem again), but how do I then stop git merge complaining about it?
Once I've done that, I'll have to switch off Eclipse's feature to automatically strip trailing whitespace (so I don't run into this problem again), but how do I then stop git merge complaining about it?
I just found a way to stop git checking for "suspicious patch lines", which gets around the reason I did the whitespace cleanup in the first place.
The howto is here: http://danklassen.com/wordpress/2008/12/git-you-have-some-suspicious-patch-lines/
There's still a few problems to resolve with the patches, but hopefully it should be OK.
Thanks for all your help guys!
The howto is here: http://danklassen.com/wordpress/2008/12/git-you-have-some-suspicious-patch-lines/
There's still a few problems to resolve with the patches, but hopefully it should be OK.
Thanks for all your help guys!
Actually, if you are changing a line of code in your patch, then you should strip whitespace from that line, but not from other lines in the same file.
And that is exactly the whitespace errors that git tells you about.
And that is exactly the whitespace errors that git tells you about.
I've no idea why you're having that problem, but here's a way to fix it:
# get a list of all the patches you've committed locally
mkdir .patches
git format-patch -o .patches origin/MOODLE_19_STABLE..thenameofyourbranch
# make a new branch off the tip of stable
git checkout -b newlocalbranch origin/MOODLE_19_STABLE
# reapply all your local patches
git am .patches/*
HTH
Penny
# get a list of all the patches you've committed locally
mkdir .patches
git format-patch -o .patches origin/MOODLE_19_STABLE..thenameofyourbranch
# make a new branch off the tip of stable
git checkout -b newlocalbranch origin/MOODLE_19_STABLE
# reapply all your local patches
git am .patches/*
HTH
Penny
Oooh.... that's clever 
Hi Penny,
Thanks for your suggestion. I've tried it, but when I try and apply the patches, I get hit by the same multitude of "suspicious patch lines" as before. Am I going to have to go through and find/replace all the whitespace and tabs each time?
Thanks for your suggestion. I've tried it, but when I try and apply the patches, I get hit by the same multitude of "suspicious patch lines" as before. Am I going to have to go through and find/replace all the whitespace and tabs each time?
That is very weird. Is it just happening for one particular patch, but the rest are ok?
Maybe if it's just one patch causing the problem, you can skip it , and reapply any *real* changes that were in that particular patch manually.
Maybe if it's just one patch causing the problem, you can skip it , and reapply any *real* changes that were in that particular patch manually.
I did a search for trailing whitespace in Eclipse on the .patches folder and it showed matches in nearly every file. This is starting to become most confusing.