There are three css tags that influence this: margin, padding and border. Firefox and IE use them differently but if you have only small values difference is small.
You should edit your theme css files and find tags with .sideblock and .clearer (they are usually in styles_layout.css), best value to start is 0.
The values of margins, paddings and borders have changed in Standard themes so it is also possible that your theme is using values from Standard Theme. In that case you can copy tags to your current theme but do not change Standard theme...
Dan,
There have been a fairly large number of changes in the behaviour of themes between Moodle 1.5 and 1.6. You say you are using in Moodle 1.7 an old custom theme based on standard 1.5. This means potentially running into problems in a number of areas. Could you not envisage starting your custom theme again, basing it on Moodle 1.7's standard theme?
Of course, you can always follow Mauno's advice and try to adjust your current CSS files.
Finally, if you had this theme actually visible on a live site, giving us guest access would help us help you.
Joseph
I think that you will find that IE always has a problem with rounded corners. Look at the Moodle.org front page in both IE and Firefox. Firefox has rounded corners.
About the rest of your problem - I have no idea.
Regards
David Berry