thanks to the tinymce website, im now replying to myself (partly) and for the benefit of others.
You can change the size of the popup window, which is an improvement (see http://www.tinymce.com/wiki.php/Configuration ). The change to make is in the file:
moodle/lib/editor/tinymce/lib.php
add the lines:
'theme_advanced_source_editor_height' => "500",
'theme_advanced_source_editor_width' => "720",
just prior to the existing line:
'spellchecker_rpc_url' => $CFG->wwwroot."/lib/editor/tinymce/tiny_mce/$this->version/plugins/spellchecker/rpc.php"
The width and height can be set as desired. This method leaves much to be desired (it would be better if these were in the editor settings via the moodle interface).
The file above also provides the opportunity to customise the button bars, enable the firefox (gecko) spell-as-you-type-check to work directly (as long as you dont press the spell button on tinymce tool bar) and convert the layer popups to proper windows. This last one avoids the lock out of the background, meaning I can do something else while the window is loading from the slow server! But this comes with the potential trouble of dealing with pop-up blockers. Given the small number of editor users at the site this is a reasonable compromise. Overall I feel the editor has gotten closer to being more usable for me, but it would be best if the popup code editor wasn't a popup at all and instead just appeared in the content editing textarea, so im still in the hunt for an alternative editor.