A Humboldt course designer and faculty member recently concluded a study where they taught the same class in Blackboard 6 and Moodle 1.3.
http://www.humboldt.edu/~jdv1/moodle/all.htm
The students were randomly divided into two groups on the first day, with half in Moodle, half in Blackboard.
Interesting data including grades, participation, and student satisfaction.
Joan will be presenting this at the CATS (California State University Academic Technology Staff) 2005 conference in Sacramento.
I "feel" there may be some bias here as well (please notice I say "feel" since I haven't seen the data to develop a more informed opinion). I don't think the delivery system can make that much difference -- especially between systems like Blackboard and Moodle. Now, if we were comparing online learning though email and Moodle, maybe
The difference comes from organization, pedagogy, facilitation. etc...in short, the way the system is used, not whether one system is free and one is not.
I like Moodle and I use Moodle, but I'm not ready to conclude that there is enough difference in the technology between Moodle and Blackboard for one to cause a superior online learning experience.
Just my opinion
Steve
Yes,
even if you see this as a "draw" then your conclusion should be:
"What a pitty, all that money spend for the same result..."
That is exactly how I see it, Ger... Moodle holds its own and costs zero dollars!
-- Art
(No offense intended to Art)No, it costs the same as commercial products: you buy/hire a server, you buy/hire a backup system, you hire educational support staff (you save on training at basic level because Moodle is so easy, but "loose" this money on training moodle on advanced level: 30+ modules, 10+ blocks, all these extra's )
But you save on license costs, EVERY YEAR AGAIN: So if I compare Moodle with the cheapest alternative, I can spend 10.000 euro EVERY YEAR, to hire Martin (i tried it once but this man is so busy!) or hire other companies to make local adapations that fit our local educational wishes!
I agree completely.
Steve
I agree completely Don...that is an excellent question that can be tested. But on the surface, when you look at the specific questions that were asked of students you see things like:
Did Blackboard/Moodle enhance instruction?
Technology-based activities developed problem-solving skills?
Communication tools: Interaction with classmates?
Communication tools: Interaction with instructor?
Etc.....
And on everyone of those questions (and nearly all of the questions), Moodle outperformed Blackboard. The results from the questions, seem to indicate that Moodle not only holds its own with Blackboard (which I believe it does), but that Moodle is better at allowing for the "development of problem-solving skills", "Interaction with classmates", "interaction with instructor", etc.
I just don't think that the differences in the "platforms" (the technology) can account for that. It seems to me the research question is more of a "technical question", and the results seem to go beyond answering that technical question.
Again, I teach on a Blackboard campus, but I have started using Moodle because I like it, I like the interface, I like the fact that I can do everything (or nearly everything) that I can do in Blackboard with an open source system. I will continue to use Moodle and am teaching public schools personnel how to incorporate it in their schools. I would like to see our campus move away from Blackboard and to Moodle, but that is a battle I'm not willing to fight at the moment...and I don't think I could fight that battle on the grounds that it would allow faculty to create a superior learning environment.
I can't say that Moodle allows me to create a "superior" online learning environment. I can teach online courses just as effectively in Blackboard as Moodle, but I can do it in Moodle for free and if I want to customize something, with a little knowledge I can do that.
I think the study is very interesting...I'm just a little suspecious of the results when I see Moodle being judged as superior in things like "developing problem solving skills"....whatever the teacher is doing in Moodle to develop those skills, can be done in Blackboard...in my opinion.
Steve
Steve wrote:
...when you look at the specific questions that were asked of students you see things like:
The results from the questions, seem to indicate that Moodle not only
holds its own with Blackboard (which I believe it does), but that
Moodle is better at allowing for the "development of problem-solving
skills", "Interaction with classmates", "interaction with instructor",
etc.
Yes, you are exactly right. I think the first two above questions are particularly inappropriate questions to be asking on a survey of student impressions. To answer those questions a different research design would be necessary. So all of us should beware of making too many conclusions from the study.
At least we can say, "A teacher and her 35 students, all new to Moodle and Blackboard, found that a commercial LMS (BB) and an open source LMS (M) were both able to deliver similar functionality with slightly higher levels of satisfaction amongst the Moodle-using students". Would you agree with that conclusion?
Please tell us how the presentation goes and pass on any further documents. And a big thanks to all the Humboldt State folks for all the new modules, functions and upgrades we are getting in version 1.5! Ditto...I think it is a good study as well....just need to tighten-up any conclusions that may jump to show the "technology" results in better problem-solving skills....like Don pointed out, the results should focus on "satisfaction". Just my opinion.
Steve
Don,
Yes, I think you summarized this very well...that conclusion could easily be supported by the respones. "Higher levels of satisfaction" seems to be clearly evident.
Steve
We are sort of doing it now, but not a same-semester comparison.
I am producing a very involved, upper-level capstone course. I run it on my home computers because the university wouldn't allow us to install Moodle on their servers.
Over the past 6 years this course has resided on WebCT, Blackboard and now Moodle. This course has always been used as an ongoing research project into distance learning and autonomous learning. It must be understood that this course is never advertised as a "distance learning course" in the university catalog. The students learn of this on the first day when we meet in a physical classroom.
In the past, we have had a 50% student drop rate, each semester. This semester our drop rate was under 25%. All content is the same, what we changed was the delivery system (using Moodle now) and we added in the option of attending the 5 live classes either online with our N*E*W system or in a physical classroom. The online classroom attendance has been very low.
The students who have previous experience with WebCT or Blackboard have almost universally accepted the Moodle system as a better way of doing things. The students without prior DL experience comment on how easy and intuitive things are. In the next 3 weeks we will be doing a true survey with the students addressing all of these issues.
Is Moodle perfect? Not at all. I find it frustrating and clumsy the way things are displayed at times. But considering that the course has been running 24/7 for 4 months, without a single software related interruption is amazing.
Dave
Ok. I get that the concensus is that Moodle holds its own or perhaps even edges Blackboard out a little. But this begs the business questions that any Administrator is going to ask:
1) What is the true cost of installing, maintaining the software and supporting the users?
2) If we decide to switch to Moodle, what will it cost to convert all of our existing courses?
We need clear answers to these questions to make the case for switching (I think. Am I wrong?).
claude
From my perspective the answer to question 1) is more or less moot. There may be some variation in the costs of maintenance dependent on the system interfaces and level of customisation but overall I tend to regard maintenance costs as comparable across comparable application software.
The 2nd question largely depends on the format of your course materials. Late last year we migrated from a legacy system to Moodle for 6000+ courses supporting 35,000 students.
We didn't migrate any forum postings - allowing access into the old online campus for a set period. Unfortunately we didn't have our courses as SCORM compliant but still the migration exercise still went relatively smoothly.
All the best,
Richard
Richard,
Good points.
I'm ok with assuming that the costs of supporting any CMS are going to be about the same (unless your talking about one that is technically challenging). However, the waters are somewhat muddied by the fact that while Humboldt is paying $8.600/year for it's Blackboard license, there IS some maintenance and technical support included in that. So the full compliment of maintenance and technical support that Humboldt receives = Bb M&S plus internal M&S.
So the question remains, is it going to cost Humboldt $8,600/year to maintain and support Moodle? Or will there be SOME savings over the Blackboard costs to account for the pure cost of the License (the cost that is not maintenance or support)?
Also, as far as #2 goes, you say the migration went relatively smoothly, but how much did it cost?
I find that if we assign internal people to a project, such as a Moodle migration, and don't calculate the proportion of their salaries that should be attributed to that project, then we are adding yet another source of mud, making it difficult to create accurate estimates of true costs. We need true costs to make our business case to administration. It's too easy for administration to shoot down our argument if we just say "It went smoothly during the pilot."
When you migrated a typical course how many people were involved? How many hours did it take? What would that proportion of their salary be? These are the type of questions that will help make the business case for Moodle (I think...)
claude
Hi Claude,
In answer to Qu1), the cost gap would theoretically be the license fee - assuming all other things are equal. Bear in mind the $8,600pa for BB only gives you the basic version.
We haven't stopped there though - the NZOSVLE project now has six higher education organisations supported by a high availability cluster and technical team - so the cost of ownership drops for hardware investment, technical support etc. We've been able to drop the support costs for an institution of 2000 learners to around $US300 per month for 24*7 technical support, hardware, bandwidth etc.
For Qu2), I'd estimate approximately 3 person-months - we could accurately define this from time-sheet reports but it would vary from organisation to organisation.
I believe the business case needs to take into account a strategic shift in using FOSS - the freedom to pursue different online pedagogies. The ROI won't be just the $ but quality improvements by not being contrained by the commercial imperatives of a vendor. As an example we were motivated by being able to supoort local and Pacific languages like Maori, Tongan, and Samoan - we'd be a long time waiting for this to be supported in a commercial application. A second example is that we've customised the Assessment module to support the particular workflows of the Open Polytechnic - being a large distance learning provider.
I hope this helps - business cases can be tricky beasts - not everything is black and white.
