Yes - this kind of "head hacks" are prevented by core code of moodle 2. Scripts do not get initialized correctly.
If you want to use MathJax with all themes of your moodle 2.X.X the easiest way is to go to Admin -> Appearance -> Additional HTML -> Within HEAD
and add normal script tags with possible configuration options there. Attached an example as a text file because moodle 2 here in moodle.org is using a filter that changes double square brackets to links to Wiki and therefore some of those old posts do not show correct initialization code anymore...
If you want to use MathJax with some particular theme only you can add the script tags to your moodle 2.X.X theme layout files - other suggested (recommended) ways to use javascript do not seem to take action - like we have discussed in many, many posts before with Ravi, Colin, Marc and others in this forum (for example in http://moodle.org/mod/forum/discuss.php?d=155977 and http://moodle.org/mod/forum/discuss.php?d=171161)
When you check the code you need to know the exact path to your mathjax script files (in my example http://path/to/your/mathjax/MathJax/MathJax.js ) and decide which delimiters to use for inline maths and display maths - official mathJax 1.1 support is in
http://www.mathjax.org/docs/1.1/index.html
In my example inline maths is rendered with @i-delimiters and display maths with @d delimiters but it's just an example and does not force anybody to use these delimiters...