Good points; a generic CMS like Wordpress, Joomla, or Drupal is going to take a tonne of work to make it anywhere near as useful as an LMS. Also think about much re-inventing the wheel you'd have to do. With a mature LMS, you've already thousands of users trying to do all kinds of different things with it and, with a good developer team and contributors, there's more likely to be some really coherent, practical, and useful features that you're unlikely to think of if you starting from a CMS and adapting it.
That said, I've been playing around with a collaborative platform built on Drupal (cue jokes about Drupal's core code), called OpenAtrium: http://openatrium.com/#!/ Once you get used to the pretentious naming (e.g. pages are called "spaces") it has some interesting and potentially useful features for collaborative learning. I like the amount of autonomy and control you can hand over to participants to form groups, start their own projects, control access, keep track of what others are doing, etc. in particular. Of course, it's not an LMS and not what most people have in mind when they think of online courses. However, if you want to do some strongly social constructivist oriented learning and teaching, I think it's certainly worth considering.