Ken (and anyone reading this in the future):
It's easy, and discussed here:
https://moodle.org/mod/forum/discuss.php?d=104245#p1113262
Basically, you just need to add an "auth" column to your CSV file and populate it with your authentication method. Moodle will say that it's unsupported, it will give the users some dummy password, but they'll be able to log in with their IMAP (or LDAP) password no problem.
Thanks for the help!