Regarding your formula issue: I think you need STACK installed to overcome this problem. If I understand you correctly then, for example, you want your students to simplify $$3n + 6x - (2x + 5n)$$ ?
Well, even the answer a student could give here could be $$4x - 2n$$, or $$-2n + 4x$$. These kind of commutative/associative problems can be a bind in trying to automatically mark more advanced math/algebra questions on-line.
To overcome that kind of problem you need a computer algebra system (CAS) and that is where STACK comes in. It can also generate random variables for coefficients - which was your other requirement.
Details of STACK are here: http://stack.bham.ac.uk/wiki/
...and it's all free
Hope this helps,
Ian.