Xinha and ASCIIMathML

Xinha and ASCIIMathML

by Marc Grober -
Number of replies: 24
The code that Mauno posted here: http://moodle.org/mod/forum/discuss.php?d=96160&parent=425826 which is a development package shall we say of xinha with dragmath and Equation editor (the latter based on ASCIIMathML) seems to work very well.

I did notice that the version of ASCIIMathML that the xinha folks have been using is quite old (1.4.x, while the version now published on sourceforge and which I have been using for a moodle filter is 2.0.2.)

Are there any ASCIIMathML users out there interested in doing some testing on this? On first blush it looks like the Equation editor inserts the asciimathml.js reference at the top of the file when you edit it. This is redundant for installations where the js is used as a filter but may be appropriate where you don't want to run filters, but want to insert the mathml. Additionally, while there is some drag and drop funtionality, it does not appear to have the same construction capabilities that dragmath has and seems to focus more on providing a wysiwyg for the asciimathml text expressions.


Average of ratings: -
In reply to Marc Grober

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

Thanks for pointing out the version, Marc

I could test asciimath-2.0.2 too and study some basics of that plugin... it really needs some editing.

Edit: ...and I must clean some other code too. For example whole folder plugins/InsertSmiley can be deleted because my code is using moodle type smileys and not those 580 smileys of Xinha big grin

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
Mauno,

I posted to xinha forum about asciimathml.js. Apparently, assuming you pulled the Equation Editor from the xinha repository they are using version 1.4.x There was some discussion a couple of years ago about trying to get Pete Jipsen to update code (which Pete did with 2.0.2, now on sourceforge) but they either did not change their code to reflect update or you have older code....

In any event, I copied the latest version of asciimathml to your xinha files under htmlarea and it worked like a charm. It looks the big diff may be the addition of the backticks from a visual inspection. Transformation is swift. Different approach than dragmath, more like the mathtran editor that Johnathon Fine posted about.

In order to provide the option to avoid referencing the js and having the file just parsed by the asciimathml as a filter I think we need to identify the code that inserts the js ref and provide the option to comment that out, much like I commented out the latex tokens in the dragmath integration.

Frankly havng both tools (equation editor and dragmath) is very nice! I have used xinha in the past (dekiwiki used it extensively but was going to jump since xinha discussion was that they wold not support safari any time soon) as well as tinyMCE in drupal/civicrm and frankly, from my perspective six of one half dozen of the other.... (don't know if that translates - makes no big diff to me as a user)

Have you looked at how Equation editor might get plugged into tinyMCE? I know Glem or Matt was playing with tinyMCE and dragmath as well....

I want to pound on this a bit more and then would like to look at what options you have created for having admin or user selected editor..
In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
Looking at equation.js towards identifying the code that inserts the reference to asciimathml.js I am confused a bit with this:
 if (typeof AMprocessNode != "function")
 {
 Xinha._loadback(_editor_url + "plugins/Equation/ASCIIMathML.js", function () { translate(); });
 }
As this seems to be the only situation in which the js reference might be inserted, but I don't see how it would work to insert this:
<script type="text/javascript" src="http://www.schmoodle.alaskapolicy.net/lib/editor/htmlarea/plugins/Equation/ASCIIMathML.js"></script>
and I am wondering whether the URL should be changed to that it is of the form php CFG form.....



In reply to Marc Grober

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

I am not sure is it better to use modules or plugins in Xinha so there may be some "cross-linking". Xinha has some structures from old htmlarea and developers of Xinha have been rewritten the code - plugins and modules may have same files. I guess you might as well delete plugins and change links in modules folder to absolute addresses or use relative links like ../../../somefolder/ASCIIMathML.js

 File htmlarea.php has a huge number of comments too - and most of those can be cleaned. I have left them there if somebody needs to check code. We really could get 80% away from Xinha code...

...and yes, I think I can rebuild that asciimath plugin to TinyMCE.

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

Marc,

I made a small change to equation.js so that it does not add span class=AM tags or mathml code to moodle at all an it seems to work nice with both old and new version of ASCIIMathML.js

Equations do not get translated several times, previous equations remain unchanged and it is possible to write normal text between asciimathml code.

The only drawback is that asciimathml code (or latex) is shown as code in editor screen - in pages of moodle and in equation editing popup equations are shown as equations.

Attachment marc.gif
In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -
Here's the file attached - I will add it to my next package later next week.
In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

And attached all files of the plugin - with the latest asciimathml.js

(we may have some "cross-changes", Marc...anyway I'll be back next week with some more features)

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

Actually Dragmath does the same - it shows unfiltered latex in editor screen.

I will check also the latex version of Equation plugin and may add some drop down list for selection to dialog.html (or maybe dialog.php). The fact that asciimathml is usable only in Firefox is not good - we need to have the selection to write both latex (for all users) and asciimathml for asciimathml users.

Some asciimathml tags may also have conflicts with smileys... for example

`rootNoYes` (`root( n )( y )`)

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
Yes, Mauno, the issue with the thumbs up and down has been raised numerous times regarding asciimathml and laTex. Initially this was fixable by simply changing the characters for tu and td, or disabling.... and in 1.9 one can fiddle it in an admin screen. frankly, since it is an issue with respect to all math notations, I think the default for these should be changed in core, but I just add it to what I tell any Math user to do now.

However, another issue is the problem with square brackets. While asciimathml ducks much of this, latex uses square brackets which creates a good deal of conflict when it comes to wiki usage. I commented on this to M Ludo and co in the nwiki discussion, and I don't know whether or how this may be addressed there in Moodle 2.0

As far as issues with firefox, I think you have this up side down ;=} Actually, asciimathml creates MathML on the fly and therefore will work with any browser that can interpret MathML. That means that it works with IE, but IE needs a plugin, MathPlayer for example, to interpret the MathML. Reason is that IE does not as yet interpret the MathMl on its own (though now that Microsoft has jumped into things with their own version of MathMl this might change.....) This is all part of the transition between converting equations to gifs or to MathML. The beauty of something like asciimathml is that you need not clog your pages up with miles of MathML code.... but can simply enter a text expression which will be rendered as MathML on the fly, much as a LaTex filter renders a Latex text expressin into a gif on the fly. As I understand from Jonathan Fine, the folks at MathTrans have tried to take on bidirectional transformation and it proved extremely difficult. At present MathML is still the W3C standard, while many word processors/web composers turn equations into gifs for use on the web.

As I mentioned before I think, asciimathml as a filter will parse both asciimathml text and Latex text and hopefully will also parse OpenOffice text by this summer. Frankly, I would include it as core. dragmath doesn't parse anything.... but only creates expressions and can save those expressions; the parsing, if any is to be done, is left to a filter. Dragmath can now produce I think 5 different output types, including MathML, asciimathml and latex. There is also a moodletex option, which produces latex delimited by dollar tokens (this was done so that you can use dragmath to produce equations transparently in moodle, but also can use it to generated tokenless latex.)

Not quite sure on what your discussion on seeing the text expressions means. I guess I will figure that out when I see what your changes do.

I will fiddle with your latest files, but want to know if there is some specific version of your work I have to use them with.


In reply to Marc Grober

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

You may drop the files over those files you have - or  rename your old plugin/module folder to something else before testing. Dialog.html and equation.js have only relative paths.

The first "xinharepla" package (replacement of htmlarea) had a lot of unnecessary files and I had Equation both in plugins and modules folder (same files) and the link in htmlarea.php was linked to modules folder while theme meta.php had a link to plugins folder. I suppose you have this package so you may override files of Equation in both modules folder and plugins folder.

Since that I moved xinha to folder xinha, have cleaned code of xinha and other editors quite a lot and will update the package after a couple of days when I get the next weekly package.

Edit: I will add also the option to select xinha with asciimathml to the list of editors in that package where you can select htmlarea, tinymce, xinha or fckeditor (or other editors) - probably on Thursday. By the way - dialog.html is using your code in function preview ...

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

Marc,

I thought I could as well update the xinharepla package right away - it only took a few minutes - and you may download the cleaned package from http://korpelainen.net/xinhatest/ 

The site itself is using asciimathml code if you want to test it before updating your files. Rename previous htmlarea folder and remember the change to theme/standard/meta.php or turn asciimathml filter on.

Note that after cleaning some code and plugins Xinha package is now 50% smaller. Font color for equations is taken from asciimathml.js

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
Mauno,

I want to make sure I understand the direction you are going with theme vs filter. I think we have run the gamut on options for parsing ascimathml (general filter as is available now, reference inserted in page as EqEd does now, tagged files with filter, theme based).

Folks who use asciimathml as a filter need to ensure that all pages with asciimathml or latex (or whatever additional languages may be parsed in future) are all parsed. As I understand your comment at http://korpelainen.net/xinhatest/ what you have done is create a theme for ascimathml that parses all pages through the js.... as opposed to using the admin filter selection panel. All site pages are checked for asciimathml pertinent tokens. Is that correct?

I see what you mean about the text not being converted in the editor. This is part and parcel of the same issue I started wrestling with and why I ended up adding the extra tokens. I will try to find time to play with this this evening, but have you looked already at difference in behavior between themed inclusion and use as filter? I don't think this will be a huge issue for most..... BUT, I would like to have the details all documented when we are done so that we can change behavior if we wish. I have gone back and forth with the various functions and I am hoping Peter will think about some function changes along the following lines:

* initEqEd(): If highlighted() open the highlighted text expression in EqEd** which places examples above, text and text expressions bottom left and then runs display() on the text+text expressions and puts the result in lower right
*save() replaces whatever was chosen from editor with the text+text expressions produced by EqEd (whereupon the contents of the editor - or the including page?? is redisplayed.)
* highlighted(): Determine if any portion of editor text area is highlighted
* display(): convert tokenized text to MathML
* tokenize(): parse xhtml. Will it make it easier to simply create a small array when a page is parsed for all text expressions with yet another pop-up to edit them??
** if this flag is null maybe the entire text area, which means if there is nothing there it behaves much like it does now, but if there is something there you see the entire page??

I will study what you have done in the equation file.

The other issue of course, foreshadowed by the above, is whether or not to suck up the existing entire contents of the editor when operning EqEd so as to be able to edit existing equations..... this is the type of behavior you can see on Peter's demo pages. I don't think thsi is anything we probably want to address now, but something to thing about long terms as both equation constructors suffer from the similar issue of not being susceptible to re-edit.

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
Mauno,

Pleaase also note that there are some existing issues with asciimathml. some are addressed in the forums. an example is where a teacher looks at assignments from students (/mod/assignments/submissions.php?) if text expressions appear in the portion of the text that is presented here it is NOT converted to mathml. Don't know yet whether this would be impacted by using thematic inclusion of script or whether it is because the way the pages in these situations are composed yet........ so there are additional system questions. Have not checked whether latex filter fails here in same manner, but probably should try that this evening as well.....

I don;t have teacher perms I guess in your test site, but did a wiki page showing square bracket issue.
In reply to Marc Grober

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

teacher - teacher...

Yes - I found the wiki issue (in old wiki) and in Nwiki latex $$ \sqrt[3]{4^{2}} $$ seems to work better than in old wiki. $ $ \ s q r t [ 3 ] { 4 ^ { 2 } } $ $

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
I have not been able as yet to get around to doing Nwiki install to test math notation though I have been following ML's efforts (as I understand it a new version is due out soon), but I have been curious as to how Nwiki would address this (nowiki or other device?)
In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
Very nice! xinharepla is much slimmer!
I kept my own color changes in meta.php.... but for testing since you are zipping for quick install you might as well include meta.php in the zip.... no?
Seems a bit slower than filter, but can't be quantitative at the moment (too much coffee makes everything seem a bit slower ;=} ) and it may be that xinha is a bit slower....

A carriage return breaks backtick tokenization as well as use of quotes inside backticks. Can't recall if this is bug or feature ;=}

I did some matrix fiddling here:
http://www.schmoodle.alaskapolicy.net/course/view.php?id=2
and discovered some xml issues with dragmath/asciimathml which I will have to look at.... asciimathml uses two sets of square brackets which latex uses three to do a simple matrix, so this will have to be addressed in asciimathml.js and asciimathml.xml....

all in all very nice.

how does xinha compare to current version of kitchen sink package? Holding my breath for full deal you said will be ready later this week ;=}

In reply to Marc Grober

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

As you know I am both "latex newbie" and "asciimathml newbie" so I have lots of forum posts and code to study wink

Xinha has some timeout function that is waiting some time to load all scripts and it makes rendering editor a little slower than other three canditates. There is also that focus issue - for keyboard accessibility (?) xinha has left focus outside textarea and I try to find the best way to place focus to textarea without extra clicks (like htmlarea, tinymce and fckeditor do)

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
LOL - I only have a couple of weeks on you ;=}

I will try to clue you in where I see an issue, but as I just discovered with respect to matrix notation, this is a landscape full of surprises. And much of what is at issue here arguably has to do with the antecedents to html5

I am actually a tinyMCE fan, but I think xinha is a good candidate to be included in a late 1.9.x because it offers a bridge of sorts for users, even if it is a bit slow.
In reply to Marc Grober

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

While testing TinyMCE in IE I noticed one thing that is very strange. AsciiMathMl filter was not on but the files of filter were in folder filter. For unknown reason moodle reads also the hidden asciimathml filter and shows in IE the message about AsciiMathML and disturbs other source code. After deleting the filter everything was normal again. I cleaned cache and tested this twice.

The filter causes trouble to all such code that can be interpreted to be asciimathml code - also some latex added with dragmath gets translated to other form in Firefox - it might be better to create a theme/editor/user/course level switch to show asciimathml = to load file AsciiMathML.js than use the filter.

It can be done with one new variable or editor selection with/without asciimathml.

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
I will have to test your claim, but I have my doubts here.... please double check the pages you think have been interpreted by the filter while it was not on. The recent versions of EqEd actually inserted the asciimathml.js reference at the top of the edited page via meta.php. I thought your idea of just putting the ref in the theme was OK, but then we need a simple way for the Moodle admin who is not experienced to turn this on..... I don't know enough about the nominal structure of "filter" in moodle to argue one way or the other, i.e. is asciimathml really a filter. I can tell you that it has been converted to php, and based on that I thought that it might be appropriate to look at bringing it into moodle as a core filter. Of course, then we also have to address drift between Peter's code and the support of the php derivative, unless Moodle seeks to take that on....

The warning about IE is actually built in the asciimathml.js so you would see that unless you commented it out in one version and not another.... but the filter behavior may be linked to some recent complaints about it not working in some instances.... but that is very strange in that without the meta.php reference, what would be invoking the filter? I will try to find some time to experiment, but now that you seem to have cleaned out the filter, why don;t you stick the filter back in, but alter the comment code in the js so that you can positively differentiate and see if you can pinpoint what is going on?



In reply to Marc Grober

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

I tested this filter issue with my "multi-editor-package" where theme meta.php has no links to asciimathml.js and editor functions have been taken from weblib.php to separate files for each editor. I doublechecked both source code and integration files and if the filter files were in folder /filter moodle got asciimathml.js warning in ie although it should not have been even loaded...maybe I'll find the reason later.

Anyway - I have been building this full package in small steps and will try to combine the best parts of tinymce, fckeditor and xinha. Combining htmlarea and xinha was easy because xinha was originally htmlarea but basicly all of these editors use same kind of javascript. Now when "prototypes" are ready the next step is to make configuration of editors more flexible both for editors and their plugins but it may take some time. This equation plugin is a really big challenge among other "routine work" but for an amateur coder like me it may be more or less slow progress.

I have already got all "extras" of moodle to work in tinymce and hopefully get them to work in fckeditor too. In fact fckeditor may be even more flexible to configure than tinymce - I just happened to start from tinymce and took xinha before fckeditor. Tinymce is still my favourite but these other editors have some such features that work differently than in tinymce or do not exist in tinymce.

Mathieu is creating the core integration of the next default editor and it may use more the lib file based method that Janne used for integrating htmlarea.

You can download the latest version of Nwiki (from Ludo's team) from:

http://morfeo.upc.es/crom/course/view.php?id=4&edit=1&sesskey=sidpcGIZbv

I have installed it to my editor test sites and it is a lot better than old wiki - and can handle latex.

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Mauno Korpelainen -

It must be the asciimathml  filter itself that is not working as it should. I installed a fresh copy of moodle 1.9.1 latest and if files of asciimathml filter are in folder /filter asciimathml is loaded no matter if asciimath filter is on or off in administration menu. But this does not happen for example to tex filter - if tex filter is off no latex is rendered. I cleared cache and page history + tested this with Opera too with same result.

It is not a big problem because asciimathml.js can be loaded for example from theme meta.php or in my editor test package from editor meta.php.

In reply to Mauno Korpelainen

Re: Xinha and ASCIIMathML

by Marc Grober -
by the way, there are duplicates of some of the files as well.... thousands and thousands of files.... ;=}