Moodleform validates select inputs on the server-side, using the list of valid option values. (So you don't need $mform->setType). So, if you are only populating the select using Ajax, then that won't work.
There are various ways to do this:
- Make a new formslib field type.
- Hack the form definition code, so that if a value is selected in the first drop-down, then you populate the second drop-down with the correct values, etc.