Stack, MaximaPool and Maxima on a separate server

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Number of replies: 30
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

If you have not already done so, then I strongly recommend getting stack working without MaximaPool. (Do this on a test server, which could even be your desktop.)

Next, I must apologies that the MaximaPool repository is in an odd state. The master branch is a bit out of date. You really want to use the code form the multiple-pool (but there is no way for you to know that other than being told.) The good news is that that version has reasonably good documentation: https://github.com/maths/stack_util_maximapool/tree/multiple-pools/doc

If you can't work out that documentation, then please let me know, and I will try to fix it.

Also, I will try to sort out that git repository, so it is self-explantory. Anyway, that is the version that the OU has been running all year, and it has successfully processed over 500,000 questions, through several upgrades.

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Sam Moulem -

Hi,

I'm trying to use MaximaPool; I've followed the instructions on the multiple-pool branch: https://github.com/maths/stack_util_maximapool/tree/multiple-pools/doc

The Servlet status page gives me the following:

Overall pool performance

NameValue
Processes starting up:0
Processes in use:0
Total number of processes started:0

and when I try to evaluate the expression '1+1;' I get an error:


HTTP Status 500 -

Index: 0, Size: 0

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:635) at java.util.ArrayList.get(ArrayList.java:411) at fi.aalto.maximapool.PoolCoordinator.getBestMatchingPoolName(PoolCoordinator.java:158) at fi.aalto.maximapool.PoolCoordinator.getProcess(PoolCoordinator.java:179) at fi.aalto.maximapool.MaximaServlet.doProcess(MaximaServlet.java:190) at fi.aalto.maximapool.MaximaServlet.doPost(MaximaServlet.java:155) at javax.servlet.http.HttpServlet.service(HttpServlet.java:646) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) 

Any help would be welcome.

Thanks

Sam

In reply to Sam Moulem

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

It looks like you have not started any particular pools.

https://github.com/maths/stack_util_maximapool/blob/multiple-pools/doc/server%20setup.txt is the key doc.

Once you have created the folder like /var/lib/maxima/2013073001/, you need to start that pool. On the status page http://example.com/MaximaPool/MaximaPool do

  1. Scroll to the bottom and press "Re-load pool definition".
  2. That should find the configuration.
  3. Click the "Run the low-level health-check" link for that pool to verify it works.
  4. Click back to get back to the status page.
  5. Click the "Start this pool" button for the new pool.
  6. Wait a bit for the new pool to start up. If you keep reloading the status page, you should see the number of active processes go up to whatever value you set.
  7. Click the "Run the high-level health-check" button for the newly started pool to verify that that works.
  8. Click back again.
  9. Only  then try to evaluate 1+1 using the form on the status page.

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Sam Moulem -

Hi Tim,

Sorry I'm a bit slow to react. We have a serious issues with Mahara and until it's fixed I can't do anything else.

I thought I had done what you suggested but I'll try again as soon as possible.

Thanks

Sam

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Sam Moulem -

Hi,

I looked at https://github.com/maths/stack_util_maximapool/blob/multiple-pools/doc/server%20setup.txt  in the first place; without it I wouldn't have been able to setup the pool.

I tried, again, to start the  pool to no avail.

Thanks anyway

I've got one question:

The set up instructions refer to '/var/lib/maxima/' (the path directory.root)

That directory can easily be changed to reflect our installation.

Best

Sam

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Sam Moulem -

Ignore my previous post.

I think I overlooked something in the PATH in process.conf; a small mistake.

Thanks again

Sam

In reply to Sam Moulem

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

It is probably the case that if you don't get you .conf files right, then the error messages are not as helpful as they could be. (Can you confirm that?)

I fear that fixing that is unlikely to be a high priority. Sorry.

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Sam Moulem -

Yes and no. It's hard to figure out what went wrong at first however the Servlet summary page gives interesting information and it helps when double checking  settings.

In reply to Sam Moulem

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Yes. All the information on the servlet summary pages, and the high- and low-level healthchecks, are there becuase I needed them myself to get things working on our servers.

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Sam Moulem -

Hi,

We actually never got the 'multiple-pools' working. We have stuck to maxima optimized so far.

When doing the low level health check we end up with 'Timeout!'.

Evaluating '1+1;' times out too.

My guess is, it is related to 'process.started = ' and 'process.ready = ' in process.conf.

I think we get them wrong; even though I'm not sure.

What actually they should be equal to.

Thanks

Sam

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Wesley Richards -

Hi,
please excuse that I’m meddling here.
But we also want to install Maxima on a dedicated server and access it from different Moodle systems.

These Instructions I've found:

https://github.com/maths/stack_util_maximapool/blob/master/README.md

Unfortunately I have no experience with Tomcat or Ant and the manual is not very detailed.
Is there somewhere more precise information on how to install Tomcat, Ant and Maxima integrates with MaxmiaPool.

And general a question.
Is it a good idea to install Maxima on its own server if you have no experience with Tomcat?
Is it perhaps easier to install Maxima on any Moodle server?

Please excuse my english, i hope you understand what I mean.

Thanks,

Wesley

In reply to Wesley Richards

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Sorry, it is not obvious, but it is better to use the version on the multiple-pools branch.

There is even better documentation there (I hope) https://github.com/maths/stack_util_maximapool/tree/multiple-pools/doc.

It should not be that hard to get Tomcat installed and running by following the instructions you can find online.

I guess I could save you the effort of compiling the code. Attached is a .war file that you can just drop into Tomcat's webapps folder.

The only slightly irritating thing is that like many Java applications, the configuration gets complied in. I used the standard example configuration file https://github.com/maths/stack_util_maximapool/blob/multiple-pools/doc/servlet.example.conf. However, I strongly suggest you change the password. You may also need to change the path name.

You can do that by taking advantage of the fact that a .war file is a .zip file in disguise:

  1. Rename MaximaPool.war -> MaximaPool.zip.
  2. Unzip it.
  3. Find the servlet.conf file in there, and edit it. (It is in WEB-INF/classes.)
  4. Re-zip the file. (Make sure you don't gain or lose any path name components in the new zip file, compared to the old one. The two folders WEB-INF and META-INF should be at the top level of the zip file, I think.)
  5. Rename back to MaximaPool.war.
  6. Use that.
Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Wesley Richards -
Hi Tim, Thank you for the info. I'll try next week. best regards Wesley
In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Kilian Singer -

Dear Tim,

I am working with Wesley on the realization of a exam server.
Thanks for all the help. I hope you find some time to get us running.


We both did not succeed to get the pool running. I could generate the MaximaPool.war file.

And setting up the tomcat server was also not an issue.

But on the page:

http://xxxx.xxx.xx:8080/MaximaPool/MaximaPool

we get

_________________________________________________________________________________________________________

System performance

NameValue
Servlet started:2016-11-29 11:08:58
Up time:6 minutes, 32 seconds
Active threads:14
Java free memory:87.0 MB out of 121.8 MB total memory (123.8 MB max limit)
System load:0.0 over 4 processors

Overall pool performance

NameValue
Processes starting up:0
Processes in use:0
Total number of processes started:0

Running versions

Test form

Input something for evaluation. (It must end in a ';'.)
1+1;

Timeout

Pool

Overall pool configuration

NameValue
Root directory:/var/lib/maxima
Min pool size:50
Max pool size:200
Limit on number of processes starting up:5
Maintenance cycle time:500 ms
Number of data points for averages:5
Pool size safety multiplier:3.0

Non-running versions

Check for new pool definitions

Password

_________________________________________________________________________________________________________

There is no pool I can select from the dropdown menu. It is empty.
Then independent if I reload pool definition or not. Upon pressing evaluate we get:
_________________________________________________________________________________________________________

HTTP Status 500 - <p>Index: 0, Size: 0</p><pre>java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:653) at java.util.ArrayList.get(ArrayList.java:429) at fi.aalto.maximapool.PoolCoordinator.getBestMatchingPoolName(PoolCoordinator.java:158) at fi.aalto.maximapool.PoolCoordinator.getProcess(PoolCoordinator.java:179) at fi.aalto.maximapool.MaximaServlet.doProcess(MaximaServlet.java:190) at fi.aalto.maximapool.MaximaServlet.doPost(MaximaServlet.java:155) at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) </pre>
_________________________________________________________________________________________________________
I tried to enter no password or the password as set in the servlet.conf.
Nothing changed.
Also I realized that the descriptions are not consistent:
in the githup page for the multiple-pool branch there is:
_________________________________________________________________________________________________________

    Copy the library of maxima functions distributed with STACK in $MOODLE/question/type/stack/stack/maxima to the pool folder and edit it.

    cp -R $MOODLE/question/type/stack/stack/maxima/* $MAXIMAPOOL/%%VERSION%%/.

    Copy the local maxima configuration files and maxima image to the same directory.

    cp -R $MOODLEDATA/stack/* $MAXIMAPOOL/%%VERSION%%/.
_________________________________________________________________________________________________________

whereas in the /doc/server-setup.txt it says:
_________________________________________________________________________________________________________

/var/lib/maximapool/          # This is the path directory.root from servlet.conf.
pool.conf # Configuration for the whole pool.
2013073001/ # Directory name is a STACK Maxima code version number.
process.conf # Configuration settings for this version.
maxima-optimised # The pre-compiled Maxima executable.
maximalocal.mac # Extra file loaded on start-up.
maxima/ # Copy of this version of question/type/stack/stack/maxima.
2013122500/ # ... can be mulitple versions present simultaniously.
_______________________________________________________________________________________________

I tried both layouts without success. Also it is not clear what the following line in in process.conf means:
____________________
# Command-line to launch the optimised Maxima process.
# This information is given at the end of the STACK healthcheck script. It will need to be updated
# to reflect the move of the optimised image to /var/lib/maxima/%%VERSION%%/
# This will depend on your maxima (and lisp) setup. E.g.
# /usr/lib/clisp-2.49/base/lisp.run -q -M /var/lib/maxima/%%VERSION%%/maxima_opt_auto.mem
# Do not include the "timeout" command. The servlet deals with timeouts itself.
command.line = /var/lib/maxima/%%VERSION%%/maxima-optimised

_________________
We replaced all %%VERSION%% by the version number as described in the documentation.
the maxima-optimized itself is a binary and the mentioned updated paths cannot be added there.
Although I realized that in maximalocal.mac there are plenty of references to folder locations outside of the /var/lib/maxima directory. Do we need to update them?
We closely reread all documentation and found one line which could give us a hint where things go wrong: in step 6 of the documentation it is written:
___________________________________________________________
Copy the library of maxima functions distributed with STACK in $MOODLE/question/type/stack/stack/maxima to the pool folder and edit it.
_______________________________________________________________________
What is meant by edit it. Maybe this can resolve the problem.
________________________________________________________
Finally it could be an issue with the access rights we did:
sudo chown -R tomcat8:tomcat8 /var/lib/maxima
and
sudo chown tomcat8:tomcat8 $TOMCAT/webapps/MaximaPool.war
__________________________________________________________
Sorry for the long post. I hope that you can figure out what went wrong.

Best regards
Kilian Singer

In reply to Kilian Singer

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

The status page is showing no 'Running versions' and no 'Non-running versions'. So, the pool cannot find the versions of Maxima you want it to run. (For now, I assume it is just one version.)

(And, as an aside, the reason that status page is so elaborate is that I gradually added more and more information to it as I debugged various issues I was having here at the OU. wink)

The documentation to believe is https://github.com/maths/stack_util_maximapool/blob/multiple-pools/doc/server%20setup.txt

Note that, most of the paths there, in the "layout will probably look something like this" sketch, are not fixed. The Tomcat root directory "/var/lib/tomcat8/webapps/" is determined by the patckage manager you used to install Linux.

The other paths are stored in the various config files. E.g. "/var/lib/maximapool/" is stored in "server.conf" (E.g. https://github.com/maths/stack_util_maximapool/blob/multiple-pools/doc/servlet.example.conf - Ooops! and I notice that example config has /var/lib/maxima/ instead of /var/lib/maximapool/.

As you have correctly surmised: Linux file permissions/ownership also need to be right.

I hope that gives you enough clues to debug this further. If not, please ask again.

Average of ratings: Useful (2)
In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Kilian Singer -

Hi Tim,

thanks for the quick reply. Yes, indeed I only used one version for the beginning.

I used /var/lib/maxima as it was mentioned in the documentation on the main github page.

It obviously did not change much, when I changed it consistently to /var/lib/maximapool.

Do I need to change the file

/var/lib/maximapool/maximalocal.mac

There are path reference to the the

$MOODLE/question/type/stack/stack/maxima

and

$MOODLEDATA

I only changed ownership of all files below /var/lib/maximapool/* to tomcat8.

and the MaximaPool.war in the webapp tomcat dir.

Is there something I forgot?

Is the password in the 8080:/MaximaPool/MaximaPool the same as in the

servlet.conf?


Here is my process.conf file:

### Config for this Maxima process

# As a sanity check, this must match the name of the containing folder, but not the full path.
# Typically this is the name of the version number of the current STACK, e.g. 20160922001
name = 2016091900

# Whether processes of this type should be automatically made available when
# the server starts.
auto.start = true

### Maxima related commands and parameters

# Working directory for the process
working.directory = /tmp

# Command-line to launch the optimised Maxima process.
# This information is given at the end of the STACK healthcheck script. It will need to be updated
# to reflect the move of the optimised image to /var/lib/maxima/%%VERSION%%/
# This will depend on your maxima (and lisp) setup. E.g.
# /usr/lib/clisp-2.49/base/lisp.run -q -M /var/lib/maxima/%%VERSION%%/maxima_opt_auto.mem
# Do not include the "timeout" command. The servlet deals with timeouts itself.
command.line = /var/lib/maximapool/2016091900/maxima-optimised

# Input the load command after this, if defined.
process.started = Maxima restarted.

# A file to load to maxima using load(""); set to false or empty if not needed
extra.file = /var/lib/maximapool/2016091900/maximalocal.mac

# The process has loaded when this appears.
process.ready = "/var/lib/maximapool/2016091900/maximalocal.mac"


### File handling related parameters
# these make the difference between shared filesystem installations and truly
# separate servers

# Do we handle files?
file.handling = true

# The command template to pass the current working and output directories to the process, use placeholders to mark the paths, depending on your system we may escape \-chars in the path.
# Usable placeholders:
# %WORK-DIR% the directory that won't be transferred to the client side
# %WORK-DIR-NE% use this if you do not want \-chars escaped
# %OUTPUT-DIR% the directory that will be transferred to the client side
# %OUTPUT-DIR-NE% use this if you do not want \-chars escaped
path.command = maxima_tempdir: "%WORK-DIR%/"$ IMAGE_DIR: "%OUTPUT-DIR%/"$ URL_BASE: "%PLOT-URL-BASE%"$


### Maximum lifetimes.

# This is the time that a process is allowed to take when starting up (ms).
startup.timeout = 10000

# This is the time added to the lifetime of a process when it is taken to use
# so that it wont be killed while in use (ms).
execution.timeout = 30000

# This is the lifetime given to a process (ms).
maximum.lifetime = 600000


### Estimates used to start the pool size planning.

# Estimated time to start a process of this type (ms).
startup.time.estimate = 2000

# Estimated demand for these processes (number per second).
demand.estimate = 0.1

pool.conf is unchanged to the pool.example.conf

and

servlet.conf is also unchanged just the path is now set to /var/lib/maximapool.

Any idea?

Best regards

Kilian Singer

In reply to Kilian Singer

Re: Stack, MaximaPool and Maxima on a separate server

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I recently went through this process, and you can read some of the discussion here

https://moodle.org/mod/forum/discuss.php?d=343245

Although setting it up is complex the health check scripts are very useful for working out what is happening. One of the tests is to see that your 

command.line

value really will fire up Maxima, and what version it runs

so if you go to the shell and type/paste in 

/var/lib/maximapool/2016091900/maxima-optimised

What happens? On my system I needed to run the lisp before that (and Tim explains why in the thread)

The good news is that the servlet startup seems to offer a significant performance benefit when you get it working.


Average of ratings: Useful (1)
In reply to Marcus Green

Re: Stack, MaximaPool and Maxima on a separate server

by Kilian Singer -

Dear Marcus,

thanks for the hint. This information would be relevant for all folks using clisp.

I used sbcl so the file is indeed an executable.

Best regards

Kilian Singer

In reply to Kilian Singer

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

It needs to be

/var/lib/maximapool/2016091900/maximalocal.mac

and not just that one file. Look at this bit of the set-up diagram again https://github.com/maths/stack_util_maximapool/blob/multiple-pools/doc/server%20setup.txt#L21

Although you are only setting up one version of the pool for now, you need to set it up as if it was one of many versions.

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Kilian Singer -

Dear Tim,

sorry I did not update it in the reply but in my installation I already properly put the maximalocal.mac into the 2016091900 folder as your layout suggests. I now also changed the groupownership of the $MOODLEDATA and $MOODLE directories to tomcat8. Still not working. In catalina.out there are warnings about /usr/share/tomcat8/common/... not existing. But I guess thats ok. otherwise no error messages just infos. Also it mentions that JAR was scanned for TLDs yet contained no TLDs. May this be relevant?

Maybe I should use gcl or clisp to get it running?

Wesley Richards from our computer department tried independently and had the same issue. Maybe we should move away from ubuntu/debian. What distro did you use?

Best regards

Kilian Singer

In reply to Kilian Singer

Re: Stack, MaximaPool and Maxima on a separate server

by Kilian Singer -

Dear all,

I started from a fresh install and was following all the hints I got. I guess it was important that no leftovers of wrong files are in the directories.

I have made notes which might be helpfull when added to the documentation:

Installation on debian:
sudo apt-get install tomcat8 ant openjdk-8-jdk git
(One might miss the openjdk-8-jdk, then ant build fails)

then
Download or clone the MaximaPool files, for example to $MAXIMAPOOL=/var/lib/maxima.
=>
Download or clone the MaximaPool files, for example to $MAXIMAPOOL=/var/lib/maximapool.

also:

Copy the library of maxima functions distributed with STACK in $MOODLE/question/type/stack/stack/maxima to the pool folder and edit it.
  cp -R $MOODLE/question/type/stack/stack/maxima/* $MAXIMAPOOL/%%VERSION%%/.
=>
cp -R $MOODLE/question/type/stack/stack/maxima/ $MAXIMAPOOL/%%VERSION%%/.

the process.example.conf might be changed to contain maximapool instead of maxima

If you would be willing to give me the rights I would perform the changes on the git.
I could also upload a small installation script for debian users.

Best regards and thanks for all the help

Kilian Singer


In reply to Kilian Singer

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I am glad you got there in the end, and thanks for offering the docs update.

Would you be able to do a github pull request?

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by Kilian Singer -

I guess this explains it:

https://help.github.com/articles/checking-out-pull-requests-locally/

I typically use git from command line. But the web page seems to give you a link to the right command.


Best regards

Kilian Singer


In reply to Kilian Singer

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

That document is really aimed at someone recieving a pull request, not someone making one.

The top three search results for https://www.google.com/search?q=how+to+submit+a+patch+using+a+github+pull+requset might be more relevant.

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by gerd schmidt -

Hello Tim Hunt,


I have setup MaximaPool on Ubuntu 16.04 and 17.04 and everything works fine. 

I have some problems with some meanings of

Started test string:(%i1)
Loaded test string:(%i1)

But the connetion to the platform and everything else is fine.


Now I tried to have a optimised version of maxima

Maxima --list-avail responds 5.37.2 lisp gcl

in maxima I type

    load("/var/lib/maximapool/2016122100/maximalocal.mac");
    load("/var/lib/maximapool/2016122100/maxima/stackmaxima.mac");
    load(stats);
    load(distrib);
    load(descriptive);
    :lisp (si::save-system "/var/lib/maxima-optimised") 

The last line terminates maxima and I have the new file.


As maxima in Ubuntu is called by a script I replace the file /usr/lib....../maxima with my new optimised file/image.

Now I call maxima and type

ATAlgEquiv(x^2-1,(x-1)*(x+1));

The result is

[true,true,"",""]

With the original maxima file it is not. So I assume the image contains stackmaxima etc. and assume everything is fine.

But it is not, both health tests runs in timeout. Also evaluating something.

the load(stackmaxima) advice in maximalocal.mac is commented out.

Any Idea and thanks a lot for the pool.

yours sincerly

Gerd

P.S. In my opinion optimised means the additional files are already loaded. Maybe with the pool, where the processes are started before, there is not so much difference in speed?








In reply to gerd schmidt

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I don't think you should replace the original Maxima executable.

The MaximaPool configuration file lets you give a path to the executable to run. Configure it to point to the maxima-optimised you built wherever you saved it.

maxima-optimised starts faster. As you say, for a single user, using un-optimised maxima is OK, because the STACK user does not notice the start-up time. However, when you have many students attempting a quiz at the same time, all the work of staring maxima will be happening on the MaximaPool server, and that will slow things down. For that reason it is worth optimising the start-up of each process.

All MaximaPool is really doing is running maxima like a command-line tool, piping in input, and capturing the output. So, you can directly simulate that on the command-line (even more-so if you sudo so the web-server user). That is a good way to debug problems.

Also in the MaximaPool interface (if you go there in your web browser) there are the 'healthcheck' links. The Low-level healthcheck shows you exactly what happens as maxima-pool tries to run one command using your configuration one step at a time (making a new process, not taking one form the pool). The High-level healthcheck tests what happens when MaximaPool gets a request from STACK. It takes a process from the pool and uses it.

(Those debugging tools exist because they are what were needed to get things working myself.)

I hope that helps.

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by gerd schmidt -

Hello Tim Hunt, Hello All


thanks for your help but it did not help so far. The main reason is that i don`t have any knowlegde about clisp gcl ..

From the Ubuntu batch and github I learnt I can start my "executable" with

/usr/lib/..../maxima  -eval '(cl-user::run)'   to get "normal" maxima.

with 

/var/lib/max_opti/max_optimised -eval '(cl-user::run)'

I start my own "image" and can evaluate

x+y; and also

ATAlgEquiv(x^2-1,(x-1)*(x+1));

This is possible with su tomcat8 also.

So I should work.

But with this line in my process.conf I get

#####################

type Status report

message <p>Process timed out.</p><pre>java.lang.RuntimeException: Process timed out. at fi.aalto.maximapool.MaximaProcess.waitForOutput(MaximaProcess.java:218) at fi.aalto.maximapool.MaximaProcess.<init>(MaximaProcess.java:80) at fi.aalto.maximapool.ProcessPool.makeProcess(ProcessPool.java:129) at fi.aalto.maximapool.PoolCoordinator.makeProcess(PoolCoordinator.java:199) at fi.aalto.maximapool.MaximaServlet.doHealthcheckHighLevel(MaximaServlet.java:369) at fi.aalto.maximapool.MaximaServlet.doGet(MaximaServlet.java:138) at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1504) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1460) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) </pre>

description The server encountered an internal error that prevented it from fulfilling this request.


#############

or

############

Executing command-line: /usr/lib/maxima/5.37.2/binary-gcl/maxima -eval '(cl-user::run)'

Waiting for target text: 'Maxima restarted.'

Timeout!


When I have only maxima in the command line field (that is the Ubuntu start-script) the pool is working with the "normal" maxima, but not with the optimised when (I replaced the binary as I descripted before.


Any idea?

thx


Pool configuration - 2016122100

NameValue
Maxima command-line:/usr/lib/maxima/5.37.2/binary-gcl/maxima -eval '(cl-user::run)'
File to load:/var/lib/maximapool/2016122100/maximalocal.mac
Started test string:Maxima restarted.
Loaded test string:/var/lib/maximapool/2016122100/maximalocal.mac
File handling:On
File paths template:maxima_tempdir: "%WORK-DIR%/"$ IMAGE_DIR: "%OUTPUT-DIR%/"$ URL_BASE: "%PLOT-URL-BASE%"$
Startup time limit:10000 ms
Execution extra time limit:30000 ms
Process life time limit:600000 ms
Initial estimate for the process startup time:2000 ms
Initial estimate for demand:0.1 Hz


In reply to gerd schmidt

Re: Stack, MaximaPool and Maxima on a separate server

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

When you run

/var/lib/max_opti/max_optimised -eval '(cl-user::run)'

on the command-line, what output do you get?

You need to set that as the value of the process.started setting in process.conf (which is 'Maxima restarted.' by default).

In reply to Tim Hunt

Re: Stack, MaximaPool and Maxima on a separate server

by gerd schmidt -

Hello Tim Hunt,

sorry for bothering you.

With

/var/lib/max_opti/max_optimised -eval '(cl-user::run)'

I get a working "maxima shell"

also with the "original" Maxima

/usr/bin/5.37.2//maxima -eval '(cl-user::run)'

In both there is "argument not recognized before". I admit that I have overseen this, but in both cases maxima is working.

I can get ride of the warning "argument not found when i use

...maxima -eval '(cl-user::run)' -f --

I tested both in the process.conf (there is a copy and paste in my last post) but always get the timeouts as described in my previous post.

thx

Gerd




In reply to gerd schmidt

Re: Stack, MaximaPool and Maxima on a separate server

by gerd schmidt -

Some additional information after I visited

https://github.com/maths/stack_util_maximapool/issues/14

(I also tested to start tomcat as root (tomcat8 has no homedirectory), but all rights should be ok anyway


here the working pool with batch file starting maxima


############################################################

Executing command-line: maxima

Waiting for target text: '(%i1)'

Maxima 5.37.2 http://maxima.sourceforge.net
using Lisp GNU Common Lisp (GCL) GCL 2.6.12
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.

(%i1)

Sending command:

load("/var/lib/maximapool/2016122100/maximalocal.mac");

Waiting for target text: '/var/lib/maximapool/2016122100/maximalocal.mac'

(%o1)           /var/lib/maximapool/2016122100/maximalocal.mac
(%i2)

Start-up time: 32 ms

Sending command:

1+1;
concat("--COMPLETED--k","ill--PROCESS--");

Waiting for target text: '--COMPLETED--kill--PROCESS--'

(%o2)                                  2
(%i3) (%o3)                    --COMPLETED--kill--PROCESS--

Sending command:

quit();

Total time: 34 ms


Executing command-line: maxima

Waiting for target text: '(%i1)'

Maxima 5.37.2 http://maxima.sourceforge.net
using Lisp GNU Common Lisp (GCL) GCL 2.6.12
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.

(%i1)

Sending command:

load("/var/lib/maximapool/2016122100/maximalocal.mac");

Waiting for target text: '/var/lib/maximapool/2016122100/maximalocal.mac'

(%o1)           /var/lib/maximapool/2016122100/maximalocal.mac
(%i2)

Start-up time: 32 ms

Sending command:

1+1;
concat("--COMPLETED--k","ill--PROCESS--");

Waiting for target text: '--COMPLETED--kill--PROCESS--'

(%o2)                                  2
(%i3) (%o3)                    --COMPLETED--kill--PROCESS--

Sending command:

quit();

Total time: 34 ms


############################################################

Here the bad with healthcheck 1


Executing command-line: /usr/lib/maxima/5.37.2/binary-gcl/maxima -eval '(cl-user::run)'

Waiting for target text: '(%i1)'

Timeout!


starting

/usr/lib/maxima/5.37.2/binary-gcl/maxima -eval '(cl-user::run)' ends with working maxima


###################################

process.conf for this

### Config for this Maxima process

# As a sanity check, this must match the name of the containing folder, but not the full path.
# Typically this is the name of the version number of the current STACK, e.g. 20160922001
name = 2016122100

# Whether processes of this type should be automatically made available when
# the server starts.
auto.start = true

### Maxima related commands and parameters

# Working directory for the process
working.directory = /var/lib/maximapool/2016122100/tmp
#working.directory = /tmp

# Command-line to launch the optimised Maxima process.
# This information is given at the end of the STACK healthcheck script.  It will need to be updated
# to reflect the move of the optimised image to /var/lib/maxima/%%VERSION%%/
# This will depend on your maxima (and lisp) setup.  E.g.
# /usr/lib/clisp-2.49/base/lisp.run -q -M /var/lib/maxima/%%VERSION%%/maxima_opt_auto.mem
# Do not include the "timeout" command.  The servlet deals with timeouts itself.
#command.line = /var/lib/maximapool/%%VERSION%%/maxima-optimised
#command.line = /var/lib/maximapool/2016122100/maxima
#command.line = /var/lib/max_opti/max_optimised -eval '(cl-user::run)' -f -- "$@"
#command.line = /var/lib/max_opti/max_optimised -eval '(cl-user::run)'
command.line = /usr/lib/maxima/5.37.2/binary-gcl/maxima -eval '(cl-user::run)'
#command.line = maxima



# Input the load command after this, if defined.
#process.started = Maxima restarted.
#maxima.ready.for.use = (%i1)
#maxima.ready.for.use = STACK-Maxima started
#maxima.ready.for.load = (%i1)
# The process has loaded when this appears. On old versions of STACK use the fi$
#maxima.ready.for.use = Stack-Maxima started
#maxima.ready.for.use = STACK-Maxima started
maxima.ready.for.load = (%i2)
maxima.ready.for.use = (%i2)


# A file to load to maxima using load(""); set to false or empty if not needed
extra.file = /var/lib/maximapool/2016122100/maximalocal.mac

# The process has loaded when this appears.
process.ready = /var/lib/maximapool/2016122100/maximalocal.mac


### File handling related parameters
# these make the difference between shared filesystem installations and truly
# separate servers

# Do we handle files?
file.handling = true

# The command template to pass the current working and output directories to the process, use placeholders to mark the paths, depending on your system we may escape \-chars in the path.
# Usable placeholders:
#  %WORK-DIR% the directory that won't be transferred to the client side
#  %WORK-DIR-NE% use this if you do not want \-chars escaped
#  %OUTPUT-DIR% the directory that will be transferred to the client side
#  %OUTPUT-DIR-NE% use this if you do not want \-chars escaped
path.command = maxima_tempdir: "%WORK-DIR%/"$ IMAGE_DIR: "%OUTPUT-DIR%/"$ URL_BASE: "%PLOT-URL-BASE%"$


### Maximum lifetimes.

# This is the time that a process is allowed to take when starting up (ms).
startup.timeout = 10000

# This is the time added to the lifetime of a process when it is taken to use
# so that it wont be killed while in use (ms).
execution.timeout = 30000

# This is the lifetime given to a process (ms).
maximum.lifetime = 600000


### Estimates used to start the pool size planning.

# Estimated time to start a process of this type (ms).
startup.time.estimate = 2000

# Estimated demand for these processes (number per second).
demand.estimate = 0.1

###################################

Maximalocal.mac


/* ***********************************************************************/
/* This file is automatically generated at installation time.            */
/* The purpose is to transfer configuration settings to Maxima.          */
/* Hence, you should not edit this file.  Edit your configuration.       */
/* This file is regularly overwritten, so your changes will be lost.     */
/* ***********************************************************************/

/* File generated on June 5, 2017, 10:37 pm */

/* Add the location to Maxima's search path */
file_search_maxima:append( [sconcat("/var/lib/maximapool/2016122100/maxima/###.{mac,mc}")] , file_search_maxima)$
file_search_lisp:append( [sconcat("/var/lib//maximapool/2016122100/maxima/###.{lisp}")] , file_search_lisp)$
file_search_maxima:append( [sconcat("/var/lib/maximapool/2016122100/logs/###.{mac,mc}")] , file_search_maxima)$
file_search_lisp:append( [sconcat("/var/lib/maximapool/2016122100/logs/###.{lisp}")] , file_search_lisp)$

STACK_SETUP(ex):=block(
/*    MAXIMA_VERSION_NUM_EXPECTED:0, */
MAXIMA_VERSION_NUM:0,
/*    MAXIMA_PLATFORM:"unix",  */
/*    maxima_tempdir:"/srv/www/moodledata/moodle1/stack/tmp/", */
    IMAGE_DIR:"/srv/www/moodledata/moodle1/stack/plots/",
    PLOT_TERMINAL:"png",
    PLOT_TERM_OPT:"large transparent size 450,300",
    DEL_CMD:"rm",
    GNUPLOT_CMD:"gnuplot",
/*    MAXIMA_VERSION_EXPECTED:"default", */
    MAXIMA_VERSION:"default",
    URL_BASE:"!ploturl!",
    /* Define units available in STACK. */
    stack_unit_si_prefix_code:[y, z, a, f, p, n, u, m, c, d, da, h, k, M, G, T, P, E, Z, Y],
    stack_unit_si_prefix_multiplier:[10^-24, 10^-21, 10^-18, 10^-15, 10^-12, 10^-9, 10^-6, 10^-3, 10^-2, 10^-1, 10, 10^2, 10^3, 10^6, 10^9, 10^12, 10^15, 10^18, 10^21, 10^24],
    stack_unit_si_prefix_tex:["\\mathrm{y}", "\\mathrm{z}", "\\mathrm{a}", "\\mathrm{f}", "\\mathrm{p}", "\\mathrm{n}", "\\mu ", "\\mathrm{m}", "\\mathrm{c}", "\\mathrm{d}", "\\mathrm{da}", "\\mathrm{h}", "\\mathrm{k}", "\\mathrm{M}", "\\mathrm{G}", "\\mathrm{T}", "\\mathrm{P}", "\\mathrm{E}", "\\mathrm{Z}", "\\mathrm{Y}"],
    stack_unit_si_unit_code:[m, l, L, g, s, h, Hz, Bq, cd, N, Pa, cal, Cal, Btu, eV, J, W, A, ohm, C, V, F, S, Wb, T, H, Gy, rem, Sv, lx, mol, M, kat, rad],
    stack_unit_si_unit_conversions:[m, m^3/1000, m^3/1000, kg/1000, s, s*3600, 1/s, 1/s, cd, (kg*m)/s^2, kg/(m*s^2), 4.2*J, 4200*J, 1055*J, 1.602177e-19*J, (kg*m^2)/s^2, (kg*m^2)/s^3, A, (kg*m^2)/(s^3*A^2), s*A, (kg*m^2)/(s^3*A), (s^4*A^2)/(kg*m^2), (s^3*A^2)/(kg*m^2), (kg*m^2)/(s^2*A), kg/(s^2*A), (kg*m^2)/(s^2*A^2), m^2/s^2, 0.01*Sv, m^2/s^2, cd/m^2, mol, mol/(m^3/1000), mol/s, rad],
    stack_unit_si_unit_tex:["\\mathrm{m}", "\\mathrm{l}", "\\mathrm{L}", "\\mathrm{g}", "\\mathrm{s}", "\\mathrm{h}", "\\mathrm{Hz}", "\\mathrm{Bq}", "\\mathrm{cd}", "\\mathrm{N}", "\\mathrm{Pa}", "\\mathrm{cal}", "\\mathrm{cal}", "\\mathrm{Btu}", "\\mathrm{eV}", "\\mathrm{J}", "\\mathrm{W}", "\\mathrm{A}", "\\Omega", "\\mathrm{C}", "\\mathrm{V}", "\\mathrm{F}", "\\mathrm{S}", "\\mathrm{Wb}", "\\mathrm{T}", "\\mathrm{H}", "\\mathrm{Gy}", "\\mathrm{rem}", "\\mathrm{Sv}", "\\mathrm{lx}", "\\mathrm{mol}", "\\mathrm{M}", "\\mathrm{kat}", "\\mathrm{rad}"],
    stack_unit_other_unit_code:[min, amu, u, mmHg, bar, cc, gal, mbar, atm, Torr, rev, deg, rpm, K, day, year],
    stack_unit_other_unit_conversions:[s*60, amu, amu, 133.322387415*Pa, 10^5*Pa, m^3*10^(-6), 3.785*l, 10^2*Pa, 101325*Pa, 101325/760*Pa, 2*pi*rad, pi*rad/180, pi*rad/(30*s), K, 86400*s, 3.156e7*s],
    stack_unit_other_unit_tex:["\\mathrm{min}", "\\mathrm{amu}", "\\mathrm{u}", "\\mathrm{mmHg}", "\\mathrm{bar}", "\\mathrm{cc}", "\\mathrm{gal}", "\\mathrm{mbar}", "\\mathrm{atm}", "\\mathrm{Torr}", "\\mathrm{rev}", "\\mathrm{{}^{o}}", "\\mathrm{rpm}", "\\mathrm{K}", "\\mathrm{day}", "\\mathrm{year}"],
    true)$
/* Load the main libraries. */
print(sconcat("[ STACK-Maxima started, library version ", stackmaximaversion, " ]"))$
/*load("stackmaxima.mac")$*/