Recommended setup for maxima - goemaxima or maximapool?

Recommended setup for maxima - goemaxima or maximapool?

by Stuart Anderson -
Number of replies: 13

We have used STACK now for coming up to 15 years (I think) and are currently running maxima on a dedicated VM server with MaximaPool.

Somewhere in the STACK documentation I've seen a note to say that the STACK developers now recommend running the maxima server tier with goemaxima rather than maximapool.  However, I can't now find where I saw that note as it was months ago when I read it.  However I have I've seen Tim Hunt loudly recommending goemaxima.

Can some one confirm if goemaxima is now the recommendation for setting up maxima?  And if so, what are the key benefits over maximapool that I can cite to justify the change to my stakeholders and managers?

Thanks and best wishes,

Stuart 



Average of ratings: -
In reply to Stuart Anderson

Re: Recommended setup for maxima - goemaxima or maximapool?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
To start with, if you have the server capacity available, then you can experiment with goemaxima in quite a low-risk way by setting up a new install of that, while leaving your maximapool setup in place. Then, all you need to do to switch between them is to change the URL in the STACK setting in Moodle. So, you could do back-to-back performance comparisons, and you could initially use goemaxima at low-stakes times, but stay using maximapool at high stakes times, until you are sure it is right to permanently switch.

The reasons I believe goemaxima is better (even though I am one of the two people who created maximapool 10 or more years ago) are:

1. goemaxima is implemented using more appropriate technologies for the task at hand. The wrapper around Maxima is written in Go, not Java + Tomcat, so it is a much more lightweight wrapper. Also it can use appropriate operating system mechanisms (fork) to rapidly spawn the required Maxima processes. This means that goemaxima is likely to be significantly faster / higher throughput on the same hardware.

2. The end result is packaged up as a Docker container, which is a very convenient way to build and deploy the required setup (once you have got your head around Docker!) You can take goemaxima's pre-build containers, but we chose to build our own with the particular version of STACK code, and Maxima that we want to use (built using their build files, just changing the configuration.)

3. That containerisation is good if you are deploying onto infrastructure like Kubernetes or AWS Farngate, which can give you automated scaling of the available resources as load changes.

4. goemaxima seems to be more actively maintained than maximapool.
Average of ratings: Useful (2)
In reply to Tim Hunt

Re: Recommended setup for maxima - goemaxima or maximapool?

by Stuart Anderson -
Thanks Tim, this is great and assuming I can progress things I will likely steal your words in points 1-4 to convince our Change board group.

RE: Opening paragraph.
This is exactly what I plan to do once I have figured out Docker. I've been watching a few training videos on it and it does indeed look very convenient.

Best wishes,
Stuart
In reply to Stuart Anderson

Re: Recommended setup for maxima - goemaxima or maximapool?

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
I have got to the stage where I consider Docker to be the answer before I know what the question is.
Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Recommended setup for maxima - v5.41.0 or v5.44.0 (with goemaxima)

by Stuart Anderson -
Just one more question Tim. The pre-built Stackmaxima images for docker listed at https://github.com/mathinstitut/goemaxima bundle Maxima v5.44.0 with all STACK plugin version from 4.3.10 onwards. However, the moodle-STACK installation instructions for admins still say "We currently recommend that you use Maxima 5.41.0".  

Are there any notable issues I could face if I use docker images with Maxima 5.44.0 (we currently run Maxima 5.41.0 with Maximapool).  Why are you not recommending v5.44.0?

Thanks,
Stuart
In reply to Stuart Anderson

Re: Recommended setup for maxima - v5.41.0 or v5.44.0 (with goemaxima)

by Frauke Sprengel -

At the moment, questions using "cartesian product" do not work with STACK and Maxima 5.44.

Best wishes

Frauke

In reply to Frauke Sprengel

Re: Recommended setup for maxima - v5.41.0 or v5.44.0 (with goemaxima)

by Stuart Anderson -
Great thanks, very helpful.

So I think that answers my question about what I need to do. For me to have an image which uses STACK question version 4.4.2 and runs with Maxima 5.41.0 (to ensure maximum compatibility with our current question banks), then I'm going to need to build my own docker image as that combination isn't available on docker hub.

Best wishes,
Stuart
In reply to Frauke Sprengel

Re: Recommended setup for maxima - v5.41.0 or v5.44.0 (with goemaxima)

by Stuart Anderson -
Just noticed in the github respository there is a comment about this (https://github.com/mathinstitut/goemaxima/tree/master/stack) committed 5 days ago:

"backport cartesian-product fix for 5.44 maxima version
The redifinition of cartesian-product is incompatible with newer
maxima versions. Since setting the maxima version to an older
version would technically be a breaking change, the fix is
backported to all older versions where maxima 5.44 is used instead.

The changes should also work with older maxima versions."

Can someone reassure me that using 5.44 with this patch won't lead to any unforeseen issues or am I still best to do my own image build with 5.41?  Sorry, I maintain a moodle-STACK install, but am just not familar with maxima.
In reply to Stuart Anderson

Re: Recommended setup for maxima - v5.41.0 or v5.44.0 (with goemaxima)

by Lennart Kramer -
goemaxima maintainer here (sorry for the late response, I did not see this post before)
I probably was a bit too hasty with setting 5.44 as the maxima version. The stack documentations mentions that the version is supported, but it's just not as well-tested.
However as implied by the commit, I'm not really comfortable just downgrading the maxima version as that might be an unexpected breaking change for existing users (I could probably still be persuaded otherwise if other issues spring up).
If you're OK with building your own image, using 5.41 is probably the safest choice. cartesian_product certainly works with the patch and I have personally not run into any other issues with 5.44 though.
In reply to Lennart Kramer

Re: Recommended setup for maxima - v5.41.0 or v5.44.0 (with goemaxima)

by Stuart Anderson -
Thanks Lennart. It should be OK for me to do my own build. I've now had a new VM with ubuntu, Docker and Go setup for me and hope to try out running the goemaxima setup soonish. If i experienece problems I can't resolve myself, no doubt I'll be back here seeking advice.

Thanks,

Stuart
In reply to Stuart Anderson

Re: Recommended setup for maxima - v5.41.0 or v5.44.0 (with goemaxima)

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
A meta-answer to a lot of this is (which you have probably seen me say before):

STACK lets you add 'Question tests' to your questions. (Bascially, "Suppose the student intpus this, they should get that mark".) These are the key to being able to update STACK with confidence. If you have tests, you can then make whatever change you want (new STACK version, new Maxima version, goemaxima vs maximapool) on a test server with a copy of all your questions, then use the scripts provided to run all the tests on all the questions. That is really the only way to answer your question.

Wanting to stick to the version of Maxima that all our questions were written with (5.41.0) is why we decided to build the docker container ourselves. We will have to upgrade one day, but we will pick our moment for when we do that, and all the testing and question-fixing that might entail.
In reply to Tim Hunt

Re: Recommended setup for maxima - v5.41.0 or v5.44.0 (with goemaxima)

by Stuart Anderson -
Thank you Tim.

RE: "Wanting to stick to the version of Maxima that all our questions were written with (5.41.0) is why we decided to build the docker container ourselves. We will have to upgrade one day, but we will pick our moment for when we do that, and all the testing and question-fixing that might entail."

As I've got my head into the Docker scripts today it's starting to make a lot more sense and isn't overly complicated (famous last words). I agree, I'm tempted not to rock the boat with too much change and sticking with 5.41 will make sense to begin with. It will also force me to engage with Docker more seriously from the off.

Cheers,
Stuart
In reply to Tim Hunt

Re: Recommended setup for maxima - v5.41.0 or v5.44.0 (with goemaxima)

by Stuart Anderson -
Quick Friday followup to my goemaxima efforts. I've been slightly putting off looking at it, but have bit the bullet on it this afternoon.

I'm left slightly stunned about how easy that was to get up and running. Other than a minor problem with not having my IP address mapped to a domain address (our problem and I'll request this from our server team), goemaxima setup just worked first time. The STACK health script suggests goemaxima is running faster than with my MaximaPool server, though I'll request some performance testing support to get some proper numbers out of it.

I was expecting a significantly amount of disappointment and frustration. Hopefully there will continue to be little of that ahead.

Thanks again for your advice.

--Stuart
Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Recommended setup for maxima - goemaxima or maximapool?

by Stuart Anderson -
Quick follow up to this now we have upgraded. Whilst I haven't yet deployed goemaxima to production (I will after the re-sit exams in September), we have done some performance testing. The design of this testing isn't perfect as I only had limited time from the performance test team. It looks at performance of a CentOS linux server with MaximaPool vs a new Ubuntu linux server with goemaxima. Both had identical CPU and memory (4 CPU cores plus 16GB RAM). The test ran 100 simulated students taken a test where all started within 30 seconds of each other and the exam ran for 2 hours.  I think we were comparing STACK v4.3.11. but are now on v4.4.4.

Transaction t02 server
Baseline Avg
t03 server
New Avg
change t02 server
Baseline 90th
t03 server
New 90th
change
MDL01_04_00_EnterQuizPassword 4823 4421 -402 9200 6354 -2846
MDL01_14_00_ConfirmSubmit 2324 2352 28 2440 2466 26
MDL01_02_00_Login 515 668 153 562 898 336
MDL01_01_00_MoodleHomePage 307 455 148 339 1043 704
MDL01_04_00_ClickQuiz 148 229 81 170 341 171
MDL01_03_00_ClickCourse 157 251 94 179 392 213
MDL01_15_00_FinishReview 114 192 78 135 224 89
MDL01_16_00_Logout 164 226 62 182 262 80

My interpretation of these results are that for general moodle access (not involving maxima), we see slightly worse response time due to the better performance you get from CentOS. But when the test is running we are seeing much better response times and in this test about 30% faster at start up after the exam password is entered.

Other standard testing of the built in STACK tests showed me goemaxia completing tests typically about 10%-30% quicker than with MaximaPool.

Moving to goemaxima feels like a no brainer at the moment.

Best wishes,

Stuart

Average of ratings: Useful (2)