That depends what theme you are using Paul.
Because Moodle 2 allows for different layout files to be used for different page types (and therefore has huge flexibility) some themes could have the logo on every page, some themes may have a large logo on the front page and a smaller logo on other pages, and some themes may only have the logo on the front page.
If the theme already has a logo on each page, then you can, as you say, simply replace the logo image file with a new one, purge all caches and its done.
The comments above about needing to add/change code are for a theme which does not already have the logo on each page and which can then be adapted to add the logo to each page if required.
HTH
Richard