I found them in Appearance > Additional HTML and deleted them. I also looked into the TeX binary settings to see that the libraries are simply not there. Clearing them didn't help. Then I noticed in the browser error log that mathslate's config.json on my server cannot load. It turned out json file type is not defined in my IIS. Once I have done that, the symbols pop up in Mathslate editor.
Nevertheless, I enabled TeX compatibility in MathJax filter setting and disabled TeX. I can now choose formula but I still can't enter values in the equation editor. MathJax.js, is still not loading when called in the plugin. But I can see the js file if I paste the URL in the browser. What could I still be missing?