STACK 3.0 Beta

STACK 3.0 Beta

by Christopher Sangwin -
Number of replies: 42
Picture of Particularly helpful Moodlers Picture of Plugin developers

It gives me great pleasure to announce the release of STACK 3.0 beta.

In Jan 2012 we decided to move STACK development in a new direction to develop a question type for the Moodle quiz. Since then we have made steady and significant progress. We are now ready to release beta code for testing and further development.

The code is available at

https://github.com/maths/moodle-qtype_stack/

All the key components are in place, but not all the features have been re-implemented. In particular, we have not re-implemented some of the input mechanisms. If these are important to you then please let me know.

I am very grateful to Phil Butcher of the OU for his support. Tim Hunt has also been a key collaborator, writing question behaviours for STACK and writing substantial portions of the code closest to Moodle itself. This joint work has enabled usto make some important minor changes to the Moodle core which are key to the integration of STACK fully.

I would also like to thank (1) Matti Pauna for working on the qformat_stack code which imports STACK 2 questions into STACK 3, (2) Matti Harjula for the server which increases the efficiency of the connection to Maxima, and (3) to Mikael Kurula for testing and the contribution of a Swedish translation.

STACK is an open source and collaborative project. There is still substantial work to make sure it is a useful and reliable tool. I would very much welcome further collaborations, in testing, translations and in further adding back the remaining features.

Help by with the documentation would also be most welcome, particularly with
(a) step-by-step guides and
(b) I wish I had known "...." information.
It is always difficult for those familiar with software to know what new users need to know most, and the order they need to know it.

I am also very open to suggestions of new features, although we have been very restrained in this latest round of developments!

We have placed the code on github. At this stage it would be sensible to use GIT to distribute the code, rather than downloading .zip files. There are likely to be regular updates over the next few months.

Installation instructions are

https://github.com/maths/moodle-qtype_stack/blob/master/doc/en/Installation/index.md

Best regards,

Chris Sangwin

Average of ratings: Useful (2)
In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Ben Brawn -

smile Horray! I have been playing with the alpha for the last couple of weeks and I am presenting what I have learnt to my colleagues tomorrow.

I was considering going through the xml files for the sample questions and removing the questions that throw errors on importing from 2 to 3 but I might just wait seeing a good deal went over without issues.

I haven't figured out why one has to deploy questions (I have read the documentation), my quizzes seem to be working fine without that, unless I'm missing something? Also the button to deploy questions mustn't be where it used to be becausethe documentation said it was somewhere it wasn't - unless I was just lost.

Another thing I haven't fully figured out is the Question Note (I have read the documentation), I just copy all the variable assignments that use randomisation and paste it in. So if that is all that is required and if that is useful then maybe it could/should happen automatically?

I am also looking forward to be able to use google charts (does this mean it will plot things that involve randomisation?)

I haven't played around with creating question tests yet, but I think I acomplished a crude by similar thing by adding more nodes to the partial response tree that dealt with common errors and gave corresponding feedback.

I had planned on making a screen cast of myself explaining how to use STACK and author questions and putting it on YouTube, hey you might get some more users that way.

As for what I wasn't clear on when I started 

  • I didn't know what these @'s did, at first I though you just put them around any variable you defined, but then I realised anything between them will be sent to Maxima and replaced by its output.
  • If it is CAStext, then it doesn't need @'s (since you are already using Maxima)
  • What is a 'meaningful question note' and should I just copy the variables in. 

Couple of more questions or possible bugs

  • I get 20 fails on the 'test suite for validation of student's input', is this normal?   
  • I get 3 fails on 'the suite for STACK Answer test', they seem to all relate the the failure to see 1/sqrt(x) as Algebraically Equivalent to sqrt(1/x), are you aware of this or is it something at my end.
  • The 'test connect to the CAS' doesn't seem to do anything except print the input, e.g. 3-1; prints 3-1; 

Thanks for the good work Chris and crew

In reply to Ben Brawn

Re: STACK 3.0 Beta

by Mikael Kurula -

Hi Ben!

Very nice that you're interested in STACK! To try to answer a few of your questions:

1. Deployment is connected to randomized questions. If you author a question with random parameters, then the solution of the problem might depend on the values of the parameters in an undesired way. The students get randomly chosen instances among those that you have deployed. Questions not using randomization don't need to be deployed.

2. The question note is also used with random questions. STACK uses the question note to distinguish between two random instances of the same question, e.g. when deploying. And this is also useful for reviewing after your course is done. The question note should include all random variables. Personally, I would write something like "Derivative of @expression@ is @diff(expression,x)@", if expression is a random function of x that should be differentiated. I don't know if that is good or bad, but that's how I started doing it. I'm not sure it's a good idea to make entering the question note automatically, since it might not be easy to read later, when you review.

3. You can already now plot things, with or without randomization, using the plot command.

4. The point of the automated question tests is to automatically test the functionality of your question, and try to make sure that you don't accidentally break something by editing your question. Therefore you should add at least one test case for every possible way of traversing the potential response trees (PRTs), in order to make sure that the outcome is the correct one for a given input. I don't see how you can replace this by adding cases to the PRTs, because the question tests are precisely supposed to verify that these PRT cases work properly. Indeed, you should try to predict what mistakes a student could make and add tests that capture these in the PRTs. Here the answer note is again useful for reviewing.

5. A screen cast would be very good! Please post the link if you create something! smile These Authoring quick start questions should get you up and running very quickly.

6. I didn't understand your comments on CAS text. If I'm not mistaken, then the CAS text is precisely what goes between the @ signs. See here. In the CAS Chat, you need to enclose whatever you want Maxima to evaluate between @ signs, IIRC. In your case, type in @3-1;@ and see what happens.

7. About the input and answer tests I don't know, that's a question for Chris and Tim, I guess.

Hope this can be of some help,

Mikael

In reply to Mikael Kurula

Re: STACK 3.0 Beta

by Ben Brawn -

Thanks for the help concerning plot and the CAS chat.

I guess what I was trying say about the CAS text was in the field 'Model answer' I haven't been using @ symbols and it has been working fine.

Thanks for the quick reply

In reply to Ben Brawn

Re: STACK 3.0 Beta

by Christopher Sangwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Mikael, Thanks for your reply. 

Ben, the 'test suite for validation of student's input' is as much to inform teachers what STACK considers valid!  Yes, I used as many examples as we could think of, including a whole suite used by the NUMBAS project.  Some of the "fails" are conventions.  For example, "sqr(x)" isn't recognized, but might be meaningful to some people.  Similarly, "sgn", "gcf" are not STACK functions.  It will be impossible to please everyone because there are conflicting conventions.  We will try to provide options where these can be solved. The OU have a separate input pannel project which we will hook STACK to in the future.  This is an important issue. Please send me your examples of syntax issues, and requests for changes. 

In practice, our planned reporting mechanism will generate a report of all "invalid" expressions, so we can work from cases which arise in use.

There are "failures" on 'the suite for STACK Answer test'.  I get two fails where the sqrt function interacts with complex numbers %i, which are in the AlgEquiv test and are flagged as "these test currently fail".  There are some very interesting branch cut issues here!  These are probably not going to arise in practice, but might so I've been honest.  Again, I hope these lists indcate how STACK behaves. 

What is your third failure?

Chris

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Ben Brawn -

Thanks Chris. 

 

Another thing someone commented about was the small size of the TAns field, it would be nice if it was much bigger (so you can see what you are typing easier).

Attached is a screen shot of the other failure.

Not too many issues with syntax except I though the cross product of two vectors was obtained by @express([a,b,c]~[d,e,f])@ but either STACK doesn't load that operation or I am mistaken.

Attachment Screen Shot 2012-07-05 at 11.11.21 PM.png
In reply to Ben Brawn

Re: STACK 3.0 Beta

by Christopher Sangwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Ben,

(1) Yes, larger boxes would be sensible.  I've made them bigger.

(2) I'm very surprised you have the problem wih 1/sqrt(x).  Which test is this, AlgEquiv?  This passes for me.  What version of Maxima are you using?

(3) By default STACK does not load("vect"). I'm not yet sure how to best enable teachers to indicate which extra libraries to load.  STACK loads some, but not all. 

FYI, It is very useful to set up a Maxima sandbox to help with debugging. 

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Ben Brawn -

Yeah AlgEquiv, and I was surprised too.

Maxima version 5.24 for some reason. Even though 5.27 is the default. sad

 

 

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Ben Brawn -

Hi again sorry to be a pain but for the life of me I can't figure out what I am doing wrong for this question I wrote. It returns the typical error when launching preview 

"qtype_stack_question : CAS error when instantiating the session: CAS failed to return any evaluated expressions. Please check your connection with the CAS."

I have attached the xml file if any wants to have a look, I am sure it has to be something stupid I have missed.

Thanks in advanced

In reply to Ben Brawn

Re: STACK 3.0 Beta

by Christopher Sangwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers


Dear Ben,

This is a classic questions and is precisely the kind of task STACK is designed to assess.

The source of the problem is that somehow you have a </span> tag in one of the "n" variables of your Question text. However, this does not show up in the editor, so you can't see it! I'm not surprised it was difficult to track down.  This was crashing Maxima (unsurprisingly). I will add an automatic check for this which will aid robustness. This is one downside of using an editor, rather than writing raw code. I haven't had this problem, but it was useful to trap it early. If you click on the "html" button on the editor you can see this. Or, you can cut and paste into a plain text editor and paste back.....

Fixing this, the question runs. However, there are a number of other comments I have about your question! I'm on my way to ICME today, and am giving a talk about STACK on Monday, so I'm a little rushed this morning. I'll write these up and post them as soon as I can.  They illustrate quite a few interesting things about STACK, so this turns out to be a very helpful example.

I'll also write some tips for debugging, e.g. how to see exactly what is being sent to and from Maxima when it crashes, which helps a lot in cases like this.

Thanks.

Chris

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Ben Brawn -

Cheers Chris, 

I wrote this question on a different computer and the only browser it had was IE9. I didn't understand why but it kept automatically creating blue links when I was typing. Now that I think about it, IE must have seen the @ symbol and thought it was an email address (this hasn't happened on the other 3 browsers I've used). Each time it did this I clicked the 'break link' and the 'stop automatically linking' buttons, this must be where it came from.     

In reply to Ben Brawn

Re: STACK 3.0 Beta

by Christopher Sangwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Ben,

I started to write a long reply but in the end your question, and the way you chose to author it, prompted me to expand and rearrange the "author quickstart guide".  I've added a new page.  Comments about this welcome.  I hope this helps.

Chris

 

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Yasuyuki Nakamura -

Dear Chris,

Congratulations! 

Mr. Nakahara and I would love to contribute to a Japanese translation. 

I am going to England the week after next. Unfortunately, not to Birmingham but to Sheffield this time...

Best wishes,

Yasuyuki Nakamura

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Valeria Mar -

Hello! I've been trying to test STACK following your steps in the documentation but I always get the same error in the health check. I get the error in the "Trying to connect to the CAS" and "Graph plotting" tests. The error I get is: "CAS failed to return any evaluated expressions. Please check your connection with the CAS". I've tried several changes in the configuration STACK configuration in Plugins > Question types > STACK but I keep getting the same error...

I'm attaching you the .html of the healthcheck.

In the Moodle STACK configuration I've selected Linux in platform type and, following your advice to Éva's problem in this thread, I've increased the CAS connection timeout to 15, just in case.

This is my system configuration:
CentOS 5.5
PHP 5.3.10
MySQL 5.5
Maxima 5.27 (Installed in the default path /usr/share/maxima. Works fine from the command line.)
GNUPlot 4.0 (Installed in the default path /usr/share/gnuplot. Works fine from the command line.)
Moodle 2.3.1

Thank you so much in advance!

In reply to Valeria Mar

Re: STACK 3.0 Beta

by Valeria Mar -

Solved! It was a permissions issue in Maxima's directory (/usr/lib/maxima/).

In reply to Valeria Mar

Re: STACK 3.0 Beta

by Jason Vore -

Just checking...do you remember what changes you had to make to the file permissions?  I'm running into the exact same error.

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Steve Turley -

I ran into an interesting problem when trying to get STACK to work with questions involving random numbers.  Your response to Ben about questions notes helped clear it up, but you might want to add a program check and additional documentation about it to help STACK newbies like me discover it more easily.  If you (unwisely) try to save a question having random variables without anything in the question field, the question results in a blank screen in Firefox and an internal error message in IE9.  It would be useful to have the save code check that field and print a descriptive warning about the problem.

So far, the system looks like it is going to be extremely helpful in my modern physics class this fall.  Thanks for the time and effort you all put into it!

In reply to Steve Turley

Re: STACK 3.0 Beta

by Ben Brawn -

This surprises me. it gives me an error in red each time I forgot to do it. Maybe I was on another browser. However I had to scroll to the field where the error was to actually see that there was an error. 

In reply to Steve Turley

Re: STACK 3.0 Beta

by Christopher Sangwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Steve,

I'm sorry you ran into this problem.  We certainly tried to put plenty of checks to help teachers write questions and if the question variables contain "rand" then the answer note must be non-empty.

I wonder if this problem is the CSS and browser, rather than STACK.

Chris

In reply to Steve Turley

Re: STACK 3.0 Beta

by Steve Turley -

I'm not sure why I am seeing this issue and others apparently aren't.  To help you run it down, here is what my system looks like:

STACK version: 2012070300

Moodle version: 2012062501; Release 2.3.1 (Build: 20120706)

I have a pretty vanilla system in terms of appearance and CSS.  I have been testing this using the standard theme with no extra CSS added.  I set up the quiz with 2 attempts allowed, Grading Method: highest grade, all questions on one page, and adaptive mode for feedback.

The feedback options were as follows:

For me text question, I set the question variables as follows:

b:rand(5)+1; 

The question was written as:

What is @b@+1?

[[input:ans1]]

[[validation:ans1]]
 
I left Specific feedback, General feedback, and Question note at their default values.
 
For the model answer, I used
b+1
The rest of the Input:ans1 was the default value.
 
In the potential response tree, I set Answer test to AlgEquiv, SAns to ans1, TAns to b+1, and Score to 1.  The rest were default values.  The rest of the form was with default values.
 
I tried to save this question (with the question note blank using three browsers:
 
With Google Chrome (version 20.0.1132.47 m) I got the following error when I tried to save the question:
Chrome error
With Firefox (version 13.0.1) I saw a blank screen in the browser.  I looked the the HTML behind the blank screen and there was none.
 
With IE 9 (Version 9.0.8112.16421) I saw the following:
IE9 error page
Once I added @b@ to the Question note text, the question worked fine in all browsers.
 
In case it matters, I am hosting the moodle site on an Ubuntu server running with PHP 5.3.2 and Apache 2 (Version 2.2.14).
 
To investigate the possibility of it being a theme problem, I tried switching the theme to Afterburner, Arialist, Brick, and Splash.  All had the same result.
 
In reply to Steve Turley

Re: STACK 3.0 Beta

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I can reproduce the problem. With Debugging on, you get the error message

Fatal error: Unsupported operand types in C:\Users\tjh238\workspace\moodle_head\question\type\stack\edit_stack_form.php on line 798

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: STACK 3.0 Beta

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

And given that, it was a simple fix: https://github.com/maths/moodle-qtype_stack/commit/216a70a46833f06b1750028a658945e9d4121d75

Do you know how to update your install to the latest version?

In reply to Tim Hunt

Re: STACK 3.0 Beta

by Steve Turley -

The fix works great.  Rather than get a new copy from github, I just noted your quick fix and edited the php file directly.  Thanks for the quick response and the nice work on this question!

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Steve Turley -

When I try to use a variable named psi in STACK it gives me the message unsupported keyword: PSI.  Is this a reserved variable for some reason?  (It would be really nice if it worked like the variables theta and phi do and gave me the Greek letter ψ, which we use a lot in quantum mechanics.)

In reply to Steve Turley

Re: STACK 3.0 Beta

by Mikael Kurula -

If I don't misremember, variables can be one letter only in Stack. Things with multiple letters are interpreted as functions. To type a psi, I would use latex: $\psi$, does that not work for you?

In reply to Mikael Kurula

Re: STACK 3.0 Beta

by Steve Turley -

$\psi$ doesn't work because I wanted to use it as a local variable in the Question variables.  The obvious work-around is just to use a different variable name.  There doesn't seem to be a single character limitation on variables, at least in the Question variables.  I used the variables pp and alpha without problem.  Also, if I use variables like theta and phi in my CASText, they seem to be fine (and are translated into $\vartheta$ and $\varphi$ in the displayed equations).

In reply to Steve Turley

Re: STACK 3.0 Beta

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I don't really understand this bit of the system, we will need to wait for enlightenment from Chris, but 'PSI' is listed in the $teachernotallow array in stack_cas_casstring, and it is a Maxima function/operator/reserved word or something.

So, bascially, you cannot use it.

In reply to Steve Turley

Re: STACK 3.0 Beta

by Christopher Sangwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Steve,

Yes, "psi" isn't available as a variable name because it has already been defined as 

"The derivative of log (gamma (x)) of order n+1. Thus, psi[0](x) is the first derivative, psi[1](x) is the second derivative, etc." 

If something like this doen't work, then check in Maxima if it is already defined.  

It does appear possible to re-define functions etc, but since this will have all sorts of unexpected side effects I decided it would be better not to allow question authors to do this. I appreciate in this case you need to use the letter as a variable as you might "x" etc.  I'm not sure what to do here, but I'll give it some thought as this is a perfectly sensible request. 

Chris

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Steve Turley -

Ah, I see the problem.  Perhaps we could find a synonym or something.  I can use other variables, but ψ has a long and established tradition as the wave function in quantum mechanics.  One possibility is to use case to distinguish it.  It looks like your comparison to reject the variable is case insenstive.  Could that be changed?

By the way, the more I use your package, but more helpful I find it.  Thanks so much for the time you put into developing it.  I'm a long time Mathematica user, but you've motivated me to learn enough maxima to really harness the power of these STACK questions.

In reply to Steve Turley

Re: STACK 3.0 Beta

by Christopher Sangwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Thanks Steve,

Really I'd like to find a way to provide an option for enabling Greek latters to be "variables" or "Maxima default".  \pi and \gamma are probably an exception, perhaps not.  Do you need to use \pi (as value) and \psi (as variable) at the same time?  Perhaps we need a way to specify this on a letter by letter basis, e.g.

stack_variable(psi)

might turn "psi" into an unassigned variable.  If you look at the option for i and j as sqrt(-1), variables, and also components of vectors, there are many cases. Solving one problem might break STACK for someone else!

Chris

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Steve Turley -

For my purposes, I never use \pi as a variable.  \gamma, on the other hand, is very commonly used in Special Relativity as the relativistic factor which controls how space-time coordinates scale.  I can see how, in other disciplines, this might change, however.  If you can find a way to let users turn it on and off as is done with i and j, that would be ideal.

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Steve Turley -

I think your current implementation even blocks the behavior you are trying to preserve.  Let's say the correct answer to my question was the psi function as defined in maxima.  I might want to put something like

y:psi[1](x)

as a Question variable.  Your current code prevents this.  I get the following message:

moodle error message

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Steve Turley -

As you're setting priorities for implementation, the thing I miss the most at this point is an ability to deploy n instances of a random question at a time rather than having to do them one-by-one.  For some of my questions, 25-50 deployments would be a good idea and the current system is cumbersome.

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Steve Turley -

I'm seeing an odd bug which I'm having problems reproducing, but I'm seeing it enough that I thought I'd report it here.  (Is there a better place to report bugs for this beta version?)

I am running my the questions using adaptive mode so the problem may be with the special adaptive behaviour rather than the STACK question itself.  In this mode, sometimes I will answer a question and preview mode and it displays that the answer is correct but gives zero points for the submission (see attachment as1.png).  This is odd in itself.  This question has a single note in the answer tree which is shown in the attachment as2.png.

When I go to the Summary of Attemp screen, it shows the status of the question as Not complete.

If I save the answer again and submit it in a different form it seems to update just fine.  It also doesn't happen with the same question each time.  I know these kinds of bugs are sometimes hard to run down.  Good luck with this one!

In reply to Steve Turley

Re: STACK 3.0 Beta

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Can you:

1. Export the question as Moodle XML format, and

2. Tell us an exact sequence of actions to perform on the question in the preview window, to make the problem happen.

It is fine to report bugs here. You can also use https://github.com/maths/moodle-qtype_stack/issues if you prefer.

In reply to Tim Hunt

Re: STACK 3.0 Beta

by Steve Turley -

That's a bit of a challenge since its intermittent and happens on random questions under, as yet, irreproducible conditions.  I'll keep trying to get a repeating scenario if I can.

In reply to Steve Turley

Re: STACK 3.0 Beta

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Actually, I think I have been seeing the same issue with one question. In my case, it is fine in the question preview window, but does not work when I add it to a quiz and preview the quiz.

In reply to Christopher Sangwin

Re: STACK 3.0 Beta

by Ben Brawn -

Doh! 

We moved to a supposedly better server and now everything to do with STACK is very slow, for example:

  1. To load next question in a quiz takes 5-10 seconds depending on what question it is.
  2. 5 seconds to check the student input using latex
  3. 45 seconds to 'submit all and finish' in a quiz with 4 questions

Any idea what we (the actual administrator - I am completely guilt free lol) could have stuffed up/forgot to do? (a couple of days out from semester sad)

Thanks

In reply to Ben Brawn

Re: STACK 3.0 Beta

by Mikael Kurula -

Are you using the same version of Maxima on both servers? I noticed that (at least some of) the newer versions of Maxima start up very slowly. At this time I think STACK starts a new instance of Maxima quite often and this could perhaps explain your problems.

Better ways of connecting to Maxima are being investigated. In the meantime, I solved the problem by installing an older version 5.23.2 of Maxima and optimised it as described here. See in particular the bottom of that page; if I was to set up my server today then I would certainly look into the maxima pool servlet.

Does this help at all?

In reply to Mikael Kurula

Re: STACK 3.0 Beta

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I have just been grappling with this on my Linux test server. Starting Maxima and loading the STACK library code was taking almost 10 seconds, which made STACK painfully slow. Switching to the optimised Maxima, as explained on the page Mikael linked to, reduced the load time to a small fraction of a second - and I updated those instructions as I was doing it, so they are up-to-date, and they should work.