Looking for examples of really HUGE Moodle installations

Looking for examples of really HUGE Moodle installations

by David Mudrák -
Number of replies: 16
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators
Hi folks,

there is a national-level state project starting in my country for which some e-learning system is going to be used. I would like to see Moodle being selected wink. I would like to offer some examples of already working implememtations / solutions so I can advocate better.

The project's requirements are quite hard. They plan to have some dozens of course, which is OK. But there will be up to 50.000 users registered. It is estimated that up to 25.000 users can be connected online at the same moment. The list of expected activities is not available yet but I do not think that all 25.000 user would attempt a quiz at the same time (given that quizes are the performance bottle neck).

I can imagine some solutions of this (server farms, clusters, load balancing etc.). But I need to know there there are some working instalations with similar requirments.

No technical details needed }althought more than warmly welcome!). Just let me know "here is an URL of an University and they have 30.000 users online at every single moment with no problem" smile or something like that.

Thank you very much,

David
Average of ratings: -
In reply to David Mudrák

Re: Looking for examples of really HUGE Moodle installations

by Valery Fremaux -
Hi David,

We are not running yet at full power, so I will not be of some help providing you with a real proof, but our system we are finishing to deliver to French Government was tailored to have 100,000 registered users, and 15,000 connected users.

We expect having 15% of those querying at the same time on our servers, so would the immediate handling of about 2,000 to 2,500 queries directly focussed.

Our system uses an HAProxy front end on a 4 Go Dual proc (quite enormous for being just a LB proxy) and planned three 8Gb Quad proc Web units. We have a 16Gb MySQL SQL node, wich is prepared to drive a set of complementary NDBCLUSTER mounted satellites. We yet just have one. the purpose of those is to hold the very stressed tables, in case we notice a DB overload.

In our actual measurements, the essential load is attributed to page construction algorithms, But these measurements do not yet take into account the increase of records to consider in tables such as user, or even worse role_capability assignments (such huge mappings). We will have constant look to that results as user bulks come within the system.

The system is state-wide, government scope, wich may be also an argument for you.

The global population of the project is over 400,000 people potentially being trained. The Ministery of Education annuary (LDAP aggregate) has 1,400,000 entries, just for the staff, as France has 14,000,000 scholarized people (all level).

Cheers. Hope I can help you to argue...
Average of ratings: -
In reply to Valery Fremaux

Re: Looking for examples of really HUGE Moodle installations

by HJWUCGA INC. -
Hello Valery,

I too work at an educational institution and I'd like to build an enterprise class Moodle platform.

Would you be able to share with us the technical details of your setup? It sounds like your environment is the one I'm shooting for.

Thank you.

Christopher
Average of ratings: -
In reply to Valery Fremaux

Re: Looking for examples of really HUGE Moodle installations

by David Mudrák -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators
Valery, thanks a lot for these details!

Does the project have a name and/or a site already? Was there some procedure to choose the environment to be used? If so, were there some concurrent applications beside Moodle? This might be a great argument for Moodle. TIA
Average of ratings: -
In reply to David Mudrák

Re: Looking for examples of really HUGE Moodle installations

by Valery Fremaux -
Bouncing on Robert Berntein's answer : our construction has been made actually on Catalyst studies for very big systems (up to 1,000,000 users).

The actual project is there :
http://national.pairformance.education.fr

I'm working hand in the hand for six month already with Catalyst team to implement Intel's worldwide TAO Educational program on a Moodle basis. We cross validated our estimations with Jonathan Newmann and a few other fellows in Wellington.

Pairformance is the first operational implementation of this program that will have official opening on March 4th and 5th, during the french TAO Consortium in Toulouse.
Average of ratings: -
In reply to David Mudrák

Re: Looking for examples of really HUGE Moodle installations

by Robert Brenstein -
Talk to someone at Catalyst. They know how to deal with large installs. Examples of big insitutions are Open University http://www3.open.ac.uk/media/fullstory.aspx?id=7354 and Open Polytechnic of New Zealand http://www.openpolytechnic.ac.nz/aboutus/newsandevents/elearningaward. However, I wonder what elearning program manages to get 25k of 50k users online at the same time if it is not a nationwide exam administered online.
Average of ratings: -
In reply to Robert Brenstein

Re: Looking for examples of really HUGE Moodle installations

by David Mudrák -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators
Thanks Robert,

the figures are sort of worst-scenario expectations. The target group of the project are teachers. I am afraid (as I work as a teacher myself) that these teachers will access the site during breaks between their lessons (while drinking a coffee and chatting with colleagues wink). As almost all schools have very similar lesson schedules, we can say e.g. there is so called "long break" sometimes between 10.00 and 10.20 +- 15 minutes. A lot of them will access the site after their lessons are over - let us say between 13.00 and 15.00. IMHO these are peaks when we really could face big traffic.
On the other hand, it may be shown the figures were overestimated and we have 1500 - 2500 users being online at the same moment.

Thanks for the links!
Average of ratings: -
In reply to David Mudrák

Re: Looking for examples of really HUGE Moodle installations

by Valery Fremaux -
We started indeed with these results, for 100,000 real accounts would we guess having about standard 2,500 connected users in an active phase at a time.

Some things have BTW not be taken into account by the Catalyst study :

increase of computing time due to some recursive or higher complexity algorithms ( > Log2 n) that might be used in some unoptimized functions. Although there has been a very big work in scalability and optimisation, the real effect of having 100,000 users in a table upon some JOINS might be tremendous.

We opted for a distributed organisation that segmentates the global user load on 35 MNETworked moodles. Our structure will spread out users in zoned moodle, increasing of course management complexity, but having scarse chance that ALL users of ALL nodes comme to visit ALL other nodes. So we expect that (including for national scope areas) having no more than 10,000 users within one Moodle of the array.

Cheers.
Average of ratings: -
In reply to Valery Fremaux

Re: Looking for examples of really HUGE Moodle installations

by A FRI -
Wow, these are really major projects.
große Augen

At our school, we are just planning to start using moodle.
To be able to estimate the costs I´d like to know what server space you´d recommend for a system that will have about 2000 users (not at the beginning, but maybe in a few years). I´m sure there are quite a lot of users in this forum that maintain sites of that size.

Thanx

Andi

Average of ratings: -
In reply to A FRI

Re: Looking for examples of really HUGE Moodle installations

by Valery Fremaux -
Obviously, this is a "startup" size.

I suggest that a single host 2/4Gb RAM bi or quadri would be enough. Most offers are now around 20 up to 60 euros/month fee.

The essential is that you secure your storage enough, this is the most important thing for small size systems investing much of their business within the Information System.

The end size of the system depends on the density of use :

1st case: sporadic use, well spread over time range : that is for 2000 real accounts, about 120 to 150 concurrent users as a very peak. The low range is ok : (2G/dual core).

2nd case : sporadic use, with potentially intensive periods (blended learning with full online sessions). The high range (4G/quadri)

3rd case : intensive use (constant use in all classrooms as course material support, simultaneous assessments, etc.). Here may be a real issue of tailoring the size, as usually, those intensive peaks may require over 1000% power of the "normal use".

Our experience show that php load is much higher than SQL load in common use. The real load problem will come from Web front end.

Using Haproxy on a first server can simply derivate some part of the load to an accessory server. It is quite easy to install, and to configurate, open source and free. You just need to know how to mount the moode code dir and the moodledata dir through NFS to get it serviceable by the second Web server.

A strategy that can be used, based on actual offer of hosting providers is using RPS type hosting (real proc, real ram, but shared storage : ex : http://www.ovh.com see RPS). Costs are really low and clusters can be added quite simply using Haproxy and learnign once how to map the NFS volume.

We are studying btw an implementation of service in France for hot-clustering existing moodle installations for those institutions who need such narrow time range super power increase.

Cheers. Good luck.
Average of ratings: -
In reply to David Mudrák

Re: Looking for examples of really HUGE Moodle installations

by Valery Fremaux -
Hi David,

FYI,

We will start a big measurement campain over the Pairformance project and should have some real confirmation (or infirmation) over Catalyst's estimations that suggested our architecture.

The measurement campain will shoot up to 1500 simultaneous users from a Mercury Interactive WinLoader testing tool.

Measurement will examinate the following ;

- single scenario measurement : Reader scenario (guest), Trainee scenario : non synchronous (authenticated, in course browsing, interactive activity using), Trainee scenario : synchronous (using chat, regularily messaging), Lecturer scenario (browsing course administration services, showing participants, changing derogations), Author scenario (adding/deleting activities, changing course parameters)

- mixed scenario repartition : trying to extrapolate from real logs the amount of each

- very bad cases : many messages to send by cron, reindexing search engine, backuping

- influence of distributed MNET architecture, splitting the load of the global system on smaller vmoodle (Virtual Moodling / not yet published) instances.

Cheers.
Average of ratings: -
In reply to David Mudrák

Re: Looking for examples of really HUGE Moodle installations

by Jordan Zebor -
In the San Diego Unified School District we have a Moodle installation that supports over 40,000 users ( and growing), with over 1 million page hits a day, from approximatly 2000-4000 unique users online. http://moodle.sandi.net/

We've ran into some performance bottlenecks along the way but thankfully we were able to grow into that number of users and not support them all at once. We load balance to multiple webservers using a SAN to share files, utilize object and content caching, and have a gigabit network for web server to database connections.

As we continue to scale for our district population (130,000 + teachers), I'm developing a plan to distribute the application logic over multiple database servers with the Moodle Network functionality. You may also want to think about doing the same.

Does anyone know if moodle will ever have the ability to distribute queries to allow write to the master and reads from the slaves ??


Average of ratings: -
In reply to Jordan Zebor

Re: Looking for examples of really HUGE Moodle installations

by shem pamplin -
I agree, splitting the read writes would be very cool, until then you could try a MySQL cluster.
Average of ratings: -
In reply to Jordan Zebor

Re: Looking for examples of really HUGE Moodle installations

by David Mark Weiss -
BB has a contract with our state school system, therefore, the total licensing cost to our university is somewhere in the range of $225,000 give or take, by my best estimation. The point that our support people keep making is that to switch to Moodle, may be free in terms of licensing fees, but that $225,000 gets eaten up pretty quick with the people necessary to run Moodle in an installation that has 25,000 users.

The other key point is that the university has their own proprietary student registration system, that BB interfaces with. Of course I don't know yet, what that system requires, but apparently BB offers that kind of interface service, that Moodle users would have to replicate and therefore pay for up front.

Any thoughts about this comparison?

Mark
Average of ratings: -
In reply to David Mark Weiss

Re: Looking for examples of really HUGE Moodle installations

by E. L. Cooper -

I assume that is 225,000 annually? Just common sense, never pitch moodle as the replacement system to BB until the last few months of the BB contract. On the other hand if you can get the school started using moodle on a limited basis the support is so costly argument evaporates when contracts come up.

I am also guessing that the contract to make you registration system inteface with moodle would be 5-15,000 USD depending on the registration system you have.
Average of ratings: -
In reply to E. L. Cooper

Re: Looking for examples of really HUGE Moodle installations

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators
..except as they have their own proprietary registration system, hopefully they have the internal knowledge to know how this works and be able to either set up a csv export or a database view with the information - in which case, moodle can interface with their system out of the box without requiring further customisation!
Average of ratings: -