In that last image \heartsuit is just not mimetex syntax and mimetex can't show it. If you had \Heart xyz you would get a mimetex heart followed by xyz.
The only problem is that the local distribution of latex together with Ghostscript or ImageMagic does not seem to work for what ever reason and therefore your site is using mimetex as a second option. Usually you would get error debugging messages from possible reason of failure but in this case when you have php 5.3 error reporting corrupts images...or if error reporting is taken away you don't get debugging messages ...
One option could be to install moodle 2.0 to an other folder like moodle2 with different moodledata folder and either different database or different $CFG->prefix if you use only one database and test if it works with local latex or at least gives some debugging info when debugging is enabled from administration menu. Moodle 2.0 does not have those deprecated functions of php 5.3 anymore.
These latex rendering issues are sometimes mystical - I remember well my first trials (days or moths???) to get Tex Live or MikTex working on my local test PC and finally I had to edit system paths to get the package installed - and on my test server (Debian Lenny) everything went smoothly and full TexLive package was up and running in a few hours. On shared hosts it may be hard to control all settings and permissions.