I would guess that to make this work, you would also need to add the new functions to qtype_calculated_find_formula_errors https://github.com/moodle/moodle/blob/master/question/type/calculated/questiontype.php#L1942.
Your approach seems pretty good. I can think of one possible different approach, but I am not sure it is better.
The other approach would be to define functions with names like qtype_calculated_factorial. Those can just be defined as normal functions, since the names won't clash with anything. Then, in calculate_raw, you would need to do a replace qtype_calculated_factorial -> factorial etc.
Note that, irrespective of whether you use your approach, or the one I suggest, I would recommend using a regular-expression match, something like
\b matches start of a word, so that won't match things like 'notfactorial'. Including the Open bracket in what you search for (after zero or more spaces) also reduces false positives.
Please add unit tests for this!