This has nothing to do with the tex filter. If you look at the source for your tex expression you will see that it is:
$ $\<nolink /><nolink />triangle</nolink /></nolink />~abc$ $
All those <nolink /> tags are going to confuse the TeX filter. The filter has been designed to take out <nolink> tags but not if they have this weird / at the end. Now, which part of Moodle put those in I don't know.