Syntax Highlighting filter

Syntax Highlighting filter

by Nigel McNie -
Number of replies: 77
This is the discussion thread for the syntax highlighting filter, details of which can be found in the modules & plugins database.

As of 2006/04/06, this filter should be considered beta quality. It still doesn't work very well with the WYSIWYG editor and with moodle's filtering for bad inputs such as malformed HTML and javascript XSS attacks.

In order for those things to be fixed I will need a better understanding of moodle and a good hunk of time. I'm helping to write a roleplay module for moodle at the moment so the understanding is coming, it's the time that is required smile

In the meanwhile, I will release an update to the filter about once every month with a new version of the upstream GeSHi included.
Average of ratings: -
In reply to Nigel McNie

Re: Syntax Highlighting filter

by David Scotson -

I'd made start on a wiki page for this though I've not actually used this on our own Moodle (only used it for the first time on Moodle.org 2 minutes ago).

In reply to David Scotson

Re: Syntax Highlighting filter

by Nigel McNie -
Looks good so far smile. I will improve it as I improve the filter.
In reply to Nigel McNie

Re: Syntax Highlighting filter

by Sergey Petkov -

This filter is really useful and very important. I tried to use it, but I can't understand something:

I have enabled the filter in my moodle installation.

I put my code between required tags, but the text is not colored and not formated. But if linenumbers parameter is "on", the number of each line is displayed, but no colors and formating again. I've read that there might be problems using the WYSIWYG editor, so I tried uploading an external .htm file and the result was the same.

Can somebody explain how can I see the code highlighted? In which activity/module is the code correctly displayed?

Thanks in advance!

In reply to Sergey Petkov

Re: Syntax Highlighting filter

by Nigel McNie -
Was your code PHP code? And if so, were you using the text editor only?

If it is, there is a conflict between moodle's HTML filter and PHP code. In particular, <?php marks look like the start of a HTML tag, so the filter attempts to clean it up as HTML, but discovers that it is so malformed that it just erases it.

You could try another language to see if this is the problem. This is unfortunately a rather large problem, because it's impossible to put in the <span syntax...> tags when in WYSIWYG mode, and impossible to put in <?php when not in WYSIWYG mode sad
In reply to Nigel McNie

Re: Syntax Highlighting filter

by Stuart Anderson -
I can confirm that the problem is not just confined to displaying php code.  I've tried posting up java code too and experienced the same problem.  The line numbering is in itself handy, although syntax highlighting would be VERY disirable in my institution (School of Computer Science, Uni of Manchester).

Annoyingly, after posting up a sample of code, if you subsequently edit it again, the HTML editor strips out the new line chars so the code is bunched up into one or two paragraphs.  The only work around I've found is to wrap code like this:

<pre><span syntax="java" linenumbers="yes"> 
  ... java code ... 
</span></pre>
However, the downside is that the line spacing is a little over generous, but at least you end up with line numbered code which you can edit.
In reply to Stuart Anderson

Re: Syntax Highlighting filter

by David Scotson -

It's better style to use pre tags, if I was handcoding code examples I'd use pre and then a code (in place of where the span is now). This has several benefits including the fact that it would look appropraite without any CSS and perhaps it would be best if the filter automatically created those tags.

In reply to Nigel McNie

Re: Syntax Highlighting filter

by Sergey Petkov -

I'm not talking about PHP. I tried to write a C++ code and I tried everything, but without result.

When in simple text redactor and imported as HTML page - I see line numbers, but no formating.

In Moodle's WYGIWYS editor - nothing. The editor is changing the code, and it becomes a full mess.

My primary goal is to put that code with line numbering into a test question. (even without formatting, just line numbers). Any ideas how to do it?

Here is a sample question with code:

What's wrong in following code:

enum MY_TYPE { ONE , TWO , THREE };

MY_TYPE fun () {

MY_TYPE my_var ;

my_var = ONE ;

my_var =2;

return my_var ;

}

In reply to Sergey Petkov

Re: Syntax Highlighting filter

by David Scotson -
Does it work when the HTML editor is turned off? Are you looking for an output something like the following?

What's wrong in following code:

enum MY_TYPE { ONE , TWO , THREE };
MY_TYPE fun () {
MY_TYPE my_var ;
my_var = ONE ;
my_var = 2;
return my_var ;
}
In reply to Sergey Petkov

Re: Syntax Highlighting filter

by Nigel McNie -
Okay... it seems that I should check up on these forums more often smile.

While at MoodleMoot NZ, I thought I'd have another crack at writing this module and getting around the HTMLArea conflicts. So I reimplemented the module using [code] tags instead of <span>, as that way HTMLarea won't do horrible things smile

I also took some time to make the styles look pretty similar in both Firefox and IE.

It would be really great if people could download it and give it a whirl. It's NOT compatible with the old way as the very fact that the old way uses <span> is it's biggest disadvantage. However if enough people complain it might be worth thinking about writing a little script to find all the stuff in the old form and convert it.

It also includes the latest GeSHi, 1.0.7.11.

Download here (378K)

(edit: btw, to install, unzip in your filter/ directory and enable in config. That's it!)
(another edit: I updated the wiki page with install instructions and instructions on how to embed syntax highlighting using the new format)
In reply to Nigel McNie

Re: Syntax Highlighting filter

by Stuart Anderson -
Nigel,

That's great.  I'm really impressed by this filter and since I'm the learning technlogist for the School of Computer Science in Manchester, UK, I can see your effort being really appreciated here.

I've tried the filter on my local version of 1.6 moodle with no problems yet.  I'm going to put it on our main servers and see how it performs there.

Thanks for all your work.

--Stuart.
In reply to Nigel McNie

Re: Syntax Highlighting filter

by David Scotson -

I gave it a whirl on a CVS head copy of Moodle. Installation and basic operation seem great. Some comments:

  • does Moodle's plain text format disable all filters? I'd never noticed before but thought that the plain text format would be a good fit for this module since code could have all sorts of wierdness in it that you don't want interfered with. On the other hand, it makes sense to have filters disabled for the plain text format because, in a sense, you're saying "don't mess with this text" when you choose it. Maybe this filter could also form the basis of a resource type ('highlighted source code') where the the various options (line numbers etc.) could be exposed in the setup screen?
  • leaving off the closing square bracket of the opening tag seems to confuse it. I get a warning (possibly only visible in debug mode) and the text up to the closing tag gets eaten.
  • it might be very slightly better HTML to have both pre and code tags wrapping the content though generally this won't have an actual visible impact. It matches better with the choice of [code] as your syntax too.
  • is there an inline mode where you could insert variable names or short snippets of code into running text? Probably just code without the pre would be better here (see above point).
  • the line number start option and highlight capability interact weirdly i.e. I ask for line number 4 and 5 to be highlighted, and ask it to start at line 4 then it highlights line 8 and 9.
  • the div immediately inside the li is probably unnecessary as the li acts as a block element anyway.
  • It adds a blank, but numbered line at the end if you don't close the tag on the same line as the last bit of code.
  • the padding around the code seems tailored to fit the linenumbered version and seems a bit lopsided when they aren't present.
  • I got a weird style duplication, possibly another interaction of highlight, it was only noticeable by viewing the source.

Notice the strange ',' in the middle of the style tag:

div style="font-size:110%;font-family:monospace;font-weight:normal;','font-size:110%;font-family:monospace;"

I can see this being really useful, hopefully Moodle.org can be upgraded to this new version too.

In reply to David Scotson

Re: Syntax Highlighting filter

by David Scotson -

Further to one of the above points:

I think

<pre><code class="php"> ... </code></pre>

for normal output,

<code class="php" ... </code>

for inline code snippets, and

<ol class="geshi-code-listing">
    <li><pre><code class="php"> ... </code></pre></li>
    <li><pre><code class="php"> ... </code></pre></li>
    ....
</ol>

might be the best solution. I don't think lists are allowed in pre blocks and code needs a container like pre or p but it's simpler to use pre even if the content is always going to be a single line.

You could also use list-style-position: inside to include the line number inside the highlighted section (yes, purely cosmetic I know), though having said that when I try it (using Firefox's edit CSS function) there seems to be some kind of collapse or interaction happening.

In reply to David Scotson

Re: Syntax Highlighting filter

by David Scotson -

Actually, this might be better:

<pre class="geshi"><code .....

<code class="geshi php" ....

<ol class="geshi" ....

where geshi can be used to set backgrounds/borders etc. and php for the syntax highlighting. You'd probably want to make minor changes to the list, inline and pre styles but by and large they'd probably share the same look and the actual syntax highlighting is likely to be indentical across all three.

In reply to David Scotson

Re: Syntax Highlighting filter

by Nigel McNie -
I couldn't connect to moodle.org last night to answer these questions but I'm here now...



"does Moodle's plain text format disable all filters? I'd never noticed before but thought that the plain text format would be a good fit for this module since code could have all sorts of wierdness in it that you don't want interfered with. On the other hand, it makes sense to have filters disabled for the plain text format because, in a sense, you're saying "don't mess with this text" when you choose it. Maybe this filter could also form the basis of a resource type ('highlighted source code') where the the various options (line numbers etc.) could be exposed in the setup screen?"

I don't know enough about moodle to comment here other than to say I think it's a good thing if people can post code in the wysiwyg area and have it work just the same as if they're using plain text or indeed anything else.

And I'm completely lost on the "resource type" thing, you might have to explain that one a little more smile

"leaving off the closing square bracket of the opening tag seems to confuse it. I get a warning (possibly only visible in debug mode) and the text up to the closing tag gets eaten."

I fixed that in the new version (see below).

"it might be very slightly better HTML to have both pre and code tags wrapping the content though generally this won't have an actual visible impact. It matches better with the choice of [code] as your syntax too."

<code> is not valid really, because it is an inline element and so cannot contain block level elements - such as the ordered list for line numbering.

In reality, the <pre> actually can't contain an ordered list either according to the XHTML spec. But most browsers handle it fine and the alternative is using a <div> which means that GeSHi has to insert a bunch more markup to simulate whitespace. So really it's an academic choice - both work, one is "correct", one is "faster".

While developing GeSHi I've been contacted by many people on this very issue and there's not really a lot I can do for it, at least while GeSHi is "stable" like the 1.0.X series is. There's no clean solution it seems for displaying any sort of code. Another example is this - you might think that using an ordered list for line numbers make sense, but if you try to select the code in Firefox, Firefox selects the line numbers too - which really sucks when pasting code! I've yet to come across a really clean solution to this problem...

"is there an inline mode where you could insert variable names or short snippets of code into running text? Probably just code without the pre would be better here (see above point)."

There was in the old version, I'll add it to the TODO for this one. Suggestions for tag format welcome.

"the line number start option and highlight capability interact weirdly i.e. I ask for line number 4 and 5 to be highlighted, and ask it to start at line 4 then it highlights line 8 and 9."

This is an interesting issue - it works like this in the GeSHi core, but perhaps it should be changed to work the way you describe. The only problem is that I can't break backward compatibility with old GeSHi versions...

However, I'm planning on putting out a GeSHi 1.0.8 at some point in the future that _will_ break some BC, in preperation for 1.2's release (it will be like a "transitional" version, giving people a chance to change to the new API). Maybe I could write the fix for that version and "back port" it for the moodle filter...

"the div immediately inside the li is probably unnecessary as the li acts as a block element anyway."

GeSHi puts it there deliberately, because otherwise you can't style the line numbers without having those styles affect the code (which is most problematic when you want to change the font size or style of the line numbers). Again this is a problem with the use of an ordered list, but this can't be changed...

"It adds a blank, but numbered line at the end if you don't close the tag on the same line as the last bit of code."

It shouldn't, I spent a while trying to stop that. Could you post the [code] block you used so I can test it?

"the padding around the code seems tailored to fit the linenumbered version and seems a bit lopsided when they aren't present."

As a matter of fact I did that deliberately, so they lined up ;). Perhaps that's not such a good idea... thoughts?

"I got a weird style duplication, possibly another interaction of highlight, it was only noticeable by viewing the source."

That was a bug :p. I fixed it in the new version.



Anyway, now for the new version:

Last night I couldn't connect to moodle.org (as mentioned), so I couldn't find the bugs to fix. So instead I implemented caching smile

Now the output is cached on the filesystem under the moodle data directory. In addition, you can configure a hoover for this directory - every now and then the directory can be swept for old cache files which are then removed. See the filter.php file near the top for the configuration of all this stuff. I hope to write a frontend for it later.

What would be nice is if filters could have cron... is this possible? If so the cache directory could be swept by a cron job instead of user requests. Although the cache is implemented currently so that you can set a maximum amount of time that the hoover will run and delay the user request.

Download here (277 K)

In reply to Nigel McNie

Re: Syntax Highlighting resource

by David Scotson -

Regarding a Syntax Highlighting 'resource', instead/as well as a filter: in Moodle most content is either a Resource or an Activity. It's a bit of an artificial split, but generally resources are more text for reading alone and activities more social/interactive.

Both are modular and people have created all sorts of esoteric add-ins for various purposes which, when installed, appear in the little "Add a something..." drop down lists when you're in editing mode.

Current different types of resource include:

  • text
  • html page
  • display file
  • show directory
  • link to website

So with a hypothetical Syntax Highlighting resource, you would first have an admin either install it (or switch it on if it becomes a standard part of Moodle). Then you would select 'Add code listing' from the 'Add a resource...' drop down.

This would take you, as the other resources currently do, to a page where you could give it a title, short description and write or paste in the actual content, a bit of code in this case.

The big difference would be that below that you could have a bunch of options that could get passed through to Geshi e.g. linenumbers checkbox, a drop-down for choice of language etc.

The other benefit is you could (I believe) do what the current plain text format appears to do and not bother calling any other filters before or after Geshi.

I don't think it would replace the need for filtering within longer texts, or in wikis, forum posts etc. but I think for longer pieces of code you'd be as well adding any commentary text as code comments and putting the whole thing up as a code resource.

(One final thought, you could also provide a direct download link which provides the content as an actual (un-numbered, non-highlighted) file which would prevent cut and paste weirdness and work around the numbered list bug in Firefox.)

In reply to David Scotson

Re: Syntax Highlighting resource

by Nigel McNie -
That sounds like a very interesting idea, especially when Moodle is being used in for some kind of coding course (when I went to uni we had to download "starter" code from the course website, this would be perfect for that).

I've had a quick play with the resource stuff. It seems to me like the directory resource could be copied and modified to support a hierarchy of code files rather than text/other files. And if it was done that way the resource might even be able to support uploading of a .zip file that contained a directory structure (something I couldn't find for the current directory resource - does this exist?).

However, if it was done that way then you'd end up with two types of "directory" resource - one for "code" and the other as it is currently.  Perhaps that isn't so desirable, and perhaps instead when you upload a file there could be some user interface to mark the file as code.

I think the idea of a resource is quite a cool idea, but I'm probably not the best person to decide how it should work. However, I'm quite happy to code it smile.

And a couple of reminders:



"It adds a blank, but numbered line at the end if you don't close the tag on the same line as the last bit of code."

It shouldn't, I spent a while trying to stop that. Could you post the [code] block you used so I can test it?

"the padding around the code seems tailored to fit the linenumbered version and seems a bit lopsided when they aren't present."

As a matter of fact I did that deliberately, so they lined up ;). Perhaps that's not such a good idea... thoughts?
In reply to Nigel McNie

Re: Syntax Highlighting resource

by David Scotson -

This post's a bit disjointed so I'll use bullets:

  • I had a look at the resource stuff too. I duplicated the current text resource and hacked it so that it automatically ran the contents through Geshi as if it was a php file. It all worked pretty seamlessly and I'm now convinced it's a good idea and relatively easy to do.
  • Interesting idea about the directory structure, however I'm thinking that if you're dealing with code as 'code' (i.e. a whole directory of the stuff rather than a file or two being referenced, ideally containing helpful comments) then you'd be as well downloading it into an editor, which can do the source highlighting for you. Note that Moodle supports uploading, unzipping and displaying files in several places, but not the the current directory resource. It's a little bit icky whichever way you do it (manipulating files via the web generally is) but for the show directory functionality you currently need to go into 'files' and upload then extract the zip, before going into 'show a directory' and simply choosing the newly created directory from the list.
  • similarly to the above, I thought another possible use of the filter would be spotting (via filenames) uploads of code to forums, glossaries etc.. Currently the multimedia filters turns uploaded images into <img> tags, uploaded mp3s into flash players etc. Basically these filters just rewrite the text so the output includes a link to the upleaded file and some standard HTML/Flash makes the magic happen. Maybe the Geshi filter could create an extra link/form that takes you to a hypothetical geshi_renderer.php which accepts the location of a local file as a parameter and returns the highlighted version (the Geshi demo page already does something like this, no)? I like the idea of making peer code review easier and if this filter function existed then any text containing hyperlinks to code, i.e. a manually created 'directory' list of files could be transformed in the same way. You'd need to limit it to files within the Moodle upload directories though or else every link to a php page would trigger it!
  • The extra line was only appearing when I used the Markdown format. There's a few weird interactions where e.g. if you leave a space before and after the code then escaped HTML starts appearing within the highlighted code block. I think the square bracket syntax may interfering with markdown's own syntax. Somewhat ironically, Markdown is designed to let you add raw HTML unmolested so I think the old syntax was a much better fit with markdown.
  • I looked into the numbered code listing thing in some detail, and I've come to the conclusion that the only viable way to make it work is to use javascript to delete/write the linenumber (appropriately padded with spaces) from/into a span at the start of a preformatted line with the entire code contained within <pre> tags. I've not actually got around to writing some test code (which will have to wait till after MoodleMoot) but my thoughts so far are
    • Mozilla intentionally adds linenumbers to the copy paste and considers it a feature that copying half a numbered list will result in it being renumbered automatically. But, at least on my Mac copying anything less than the whole list results in the numbers being replaced with hash marks, and an unwanted line-break appears to be getting added. Summary: broken (at least partly) by design
    • IE doesn't add the line-numbers when you copy and paste, but a) you're demographics when dealing with coders probably means a 50/50 split between IE and Firefox, and b) there are occasions when you actually want to copy and paste the numbers as well as the code so your odds are low of your users actually being able to achieve what they want, and you have zero chance if they want to do both (i.e. copy with and without linenumbers) at different times.
    • I looked at some other ideas, like Koders.com which has a two celled table with a list of numbers separated by <br> tags in the first cell. Ingenious but again only useful for copying without the linenumbers.
    • If you put the code in a standard table then Firefox lets you copy only certain parts of a table by holding down control as you drag. Again neat, but a mostly unknown feature even to the geeky users of Firefox, and not cross-browser.
    • I thought you could just display:none the numbers as appropriate, changing the class with standard javascript but both IE and Firefox cut and paste content that isn't visible as a result of this style declaration.
    • adding and removing entire tags is made difficult because each line within the <pre> doesn't have a tag each
    • so it's perhaps best to have a span that's sometimes empty/contains empty spaces and sometimes has the linenumbers (right-padded with spaces) . You could perhaps use the sometimes useless span to add linenumber anchors (though I'm not sure how you'd avoid duplicating ids if you have more than one listing per page). This has the extra benefit of letting you highlight lines simply by putting the anchor in the URL and using :target (or whatever the CSS pseudo-selector is for that). Naturally that doesn't work in Internet Explorer, at least I think it doesn't.
    • so then you'd have to specify linenumbers or not, and whether the user has the option to toggle them with javascript (bearing in mind that if they don't have javascript then they're stuck with whatever you give them).
    • maybe you could autogenerate the plain text file in javascript too, but I have no idea if that's possible.
  • regarding the lopsided padding when not using linenumbers, I'd consider putting the line-numbers outside the colored background in a 'gutter' type thing to balance it better but as that might be tricky depending on the markup I'd maybe just add extra space on the other side of the code as well to balance it.

I hope that all makes sense!

In reply to David Scotson

Re: Syntax Highlighting resource

by Nigel McNie -
"I had a look at the resource stuff too. I duplicated the current text resource and hacked it so that it automatically ran the contents through Geshi as if it was a php file. It all worked pretty seamlessly and I'm now convinced it's a good idea and relatively easy to do."

It looks like it works pretty well, I guess it just needs the GeSHi controls and to remove the selection for text format smile

"Interesting idea about the directory structure, however I'm thinking that if you're dealing with code as 'code' (i.e. a whole directory of the stuff rather than a file or two being referenced, ideally containing helpful comments) then you'd be as well downloading it into an editor, which can do the source highlighting for you. Note that Moodle supports uploading, unzipping and displaying files in several places, but not the the current directory resource. It's a little bit icky whichever way you do it (manipulating files via the web generally is) but for the show directory functionality you currently need to go into 'files' and upload then extract the zip, before going into 'show a directory' and simply choosing the newly created directory from the list."

I guess so. If people find later that they actually want this it can always be done then. It's just a case of seeing how people will actually use this first smile

"similarly to the above, I thought another possible use of the filter would be spotting (via filenames) uploads of code to forums, glossaries etc.. Currently the multimedia filters turns uploaded images into <img> tags, uploaded mp3s into flash players etc. Basically these filters just rewrite the text so the output includes a link to the upleaded file and some standard HTML/Flash makes the magic happen. Maybe the Geshi filter could create an extra link/form that takes you to a hypothetical geshi_renderer.php which accepts the location of a local file as a parameter and returns the highlighted version (the Geshi demo page already does something like this, no)? I like the idea of making peer code review easier and if this filter function existed then any text containing hyperlinks to code, i.e. a manually created 'directory' list of files could be transformed in the same way. You'd need to limit it to files within the Moodle upload directories though or else every link to a php page would trigger it!"

Okay, the way I see it there are two parts to what you say above:
  1. Where an "attachment" can be added to a forum post/glossary entry etc. etc., if the attachment is code then the GeSHi filter could be used to show the code syntax highlighted. Perhaps if the attachment is code, rather than making it a link to download the code it could be displayed like this:

    Text file file.py (27K)
    view | download

    The download link could work as if you clicked on the file name currently, and the view link would go off to the render script. Perhaps there could even be a few more icons added for different file types smile
  2. If text occurs in a forum post/glossary entry that looks like a file name of an uploaded file (assuming the file is in the same course), then links could be added to view/download that file. I'm not quite so sure how the transformation would look like because of how the file name could appear. E.g., someone may have written in the forum: "You mean the xml/foo.xml file?". The conversion in this case would have to take into account that it's inside a line of text. Maybe it could just be hyperlinked to viewing via the render script.

"The extra line was only appearing when I used the Markdown format. There's a few weird interactions where e.g. if you leave a space before and after the code then escaped HTML starts appearing within the highlighted code block. I think the square bracket syntax may interfering with markdown's own syntax. Somewhat ironically, Markdown is designed to let you add raw HTML unmolested so I think the old syntax was a much better fit with markdown."

Heh, works for one but not the other, just like before with the HTML editor vs. plain text. I will play with markdown format, and I might change the delimiter to be {code} - hopefully that will not interfere with any of the formats (and incidentally help with Stuart's wiki problems and indeed with possible future wiki integration).

Re the line numbers issues:
  • Your analysis looks pretty similar to mine - there's simply no easy way to do this sad. The koders.com method, by the way, also has another disadvantage - if the code lines happen to wrap then the numbering gets all out of kilter. In any event, using a table was tried in GeSHi and discarded for code copying and because of the large amount of extra markup added.
  • The <pre> with <span>s inside it is actually a good idea, and I've seen it done with python syntax highlighting on MoinMoin. Then there's a toggle just above the code to "add/remove line numbers". It doesn't use much markup, but unfortunately for me there's no way I can change how the numbering is done in a stable version of GeSHi - it would break far too much compatibility with other people who have integrated it. It's an idea I'd like to try for the upcoming major version of GeSHi (1.2), but that's not really stable enough for use in a filter.
  • Thus, the only reasonable solution I can think of is to provide a link by the code to "view raw code" in a separate window - this window would just show the code in a <pre> or similar.
"regarding the lopsided padding when not using linenumbers, I'd consider putting the line-numbers outside the colored background in a 'gutter' type thing to balance it better but as that might be tricky depending on the markup I'd maybe just add extra space on the other side of the code as well to balance it."

Yeah, the markup makes it pretty tricky sad. What do you mean by space on the other side of the code? You mean some right padding on the inside of the box? It's not often lines will make it that long, and even if they do, because of the <pre> that they are in, the lines will just expand off the side of the screen.
In reply to Nigel McNie

Re: Syntax Highlighting filter

by Stuart Anderson -
Hi again,

I've just spotted one minor 'gotcha', although strickly speaking it is more a case of the correct behaviour doesn't fit my particular need.  I'll get to the point...

I was thinking about the need to add code to a moodle wiki.  Unfortunately, the standard moodle convention of using square brackets for filtered content clashes with the wiki syntax of creating new pages.  So if you try to add code blocks wrapped in [code java]...[/code], the wiki page gets very confused indeed.  I investigated how easy it would be to find a work around and  I used curly braces instead, i.e. {code java}...{/code} and this seemed to work fine except that square brackets within the highlighted code block were rendered incorrectly.

Does anybody have any other suggestions as to how to use filtered content within a wiki? I really like what I see so far, but if it can work within a wiki too, all my problems will be solved!

Regards,

Stuart.
In reply to Stuart Anderson

Re: Syntax Highlighting filter

by Stuart Anderson -
I've just spotted your notes in the filter.php file:

//
// GeSHi Filter:
//
//   - GeSHi 1.0.X for now
//   - Shouldn't conflict with other filters (hard)
//   - Should be inputtable by the WYSIWYG editor(s) and the text area
//     * Perhaps match multiple types of starters and enders for this?
//     * Should prevent other filters from messing with the output (?)
//     * Should allow configurability at "runtime" but should have sane defaults
//

I'm presuming that the "Perhaps match multiple types of starters and enders for this?" would potentially solve the problem I've highlighted with using this filter in wikis?  I'll do some more digging around in the code...

--Stuart.

In reply to Stuart Anderson

Re: Syntax Highlighting filter

by Nigel McNie -
I don't know a lot about moodle wikis, but I would have thought that if they were using a parser to check the code then you could add a hook into the parser with specific start/end tags that runs it through this fiter. That's how it works for other wikis I've integrated (or have seen integrated), for example Dokuwiki.

So perhaps it's an option to hook into the wiki parser somehow? That seems the cleanest way.

And assuming that this is possible for dfwiki, then maybe we could standardise the markers for code blocks now into something that both the wiki and HTMLarea will accept (perhaps curly braces, perhaps something else), so that you will be able to use the code highlighting all over a moodle site without using different markers in different places.
In reply to Nigel McNie

Re: Syntax Highlighting filter

by Stuart Anderson -
I guess if the marker could be configued in the admin setup, this would offer the most flexibility.  I got limited success by changing the filter to look for {code} {/code}, but it had problems recognising how to handle square brackets inside the code tags.  However, I think the problem was largely down to me not understanding the PHP implementation enough.  May be you can help or point me to all the important lines of code.

--Stuart.
In reply to Stuart Anderson

Re: Syntax Highlighting filter

by Nigel McNie -
I'll point you to the lines of code, if you have success with the change then maybe it's worth making {code} standard as part of the module right now while it's not bein used so much. That note I had in the source about multiple starters/enders was actually me musing over how to get the highlighting working with the <span> marker - e.g. I was going to try highlighting if <span...> or &lt;span...&gt; were detected to handle the wysiwyg editor. But changing the delimiter makes much more sense smile

Anyway, the lines of code you need to change:

lines 105, 106:

    while (false !== ($pos = strpos($input, '[code '))) {
        $endpos = strpos($input, '[/code]', $pos);

line 112:

            $endpos = strpos($input, '[/code]', $endpos + 1);

lines 124 - 126:

        $codeblock = str_replace('\\[/code]', '[/code]', $codeblock);
       
        $parts = explode(']', $codeblock, 2);

line 130:

            $result .= '[code ' . $parts[0] . $parts[1] . '[/code]';


Note: if you change the length of the delimiter you will need to change other lines also, namely 122 and 135.
In reply to Nigel McNie

Re: Syntax Highlighting filter

by Stuart Anderson -
Hi Nigel,

Finally found time to play with the code.  I have no idea which line I missed out this time, however, the edits to change [ ... ] to { ... } worked fine so that wrapping code in {code lang} {/code} works.  However, things still go horribly wrong when posting code into a wiki page and you end up with unescaped HTML sad
I posted a message to the dfwiki people to see if they will take interoperability with filters into consideration but got no reply.  I would imagine filters are the least of their problems right now!

Thanks for your help.

--Stuart.
In reply to Stuart Anderson

Re: Syntax Highlighting filter

by David Scotson -

Does the Moodle wiki(s) not support something similar to <nowiki> which in MediaWiki for example, stops processing of text for wiki markup. It's used in the Moodle Docs Wiki for exactly this kind of thing.

edit: the efurtwiki docs appear to claim that using <code> will prevent it from messing with your text. However, I can't seem to get it to work.

Off-topic, but I'm highly amused that the wiki markup for <small> text is two µ's!

In reply to David Scotson

Re: Syntax Highlighting filter

by Stuart Anderson -
The best I could get it to do was to display the code block in preformatted format, but the syntax filter didn't filter the code part.  I also tried to look for some sort of wiki escape character so that it ignored the wiki syntax, but I also couldn't find anything that worked.

--Stuart
In reply to Stuart Anderson

Re: Syntax Highlighting filter

by David Scotson -

If syntax coloring within wikis is important for you, it might be worth looking at the newer dfwiki that is scheduled, if ready, to become the new wiki module for the 1.7 release. It apparently supports the <nowiki> tags.

I don't really use either much, but I think that the older erfurtwiki has somewhat languished in the shadow of the new dfwiki that has unfortunately taken longer to arrive than originally planned.

In reply to David Scotson

Re: Syntax Highlighting filter

by Stuart Anderson -
Thanks David,

I'll install the dfwiki and see what it is like in its current state.  If the <nowiki> tag works and doesn't break the syntax filter, then I think this is the best solution as the use of square brackets by your filter sticks to moodle conventions.  If anything, may be the moodle convention was a little short sighted with regards to the complications is causes with wikis.

--Stuart.
In reply to Stuart Anderson

Re: Syntax Highlighting filter

by Doug Shawhan -
Has anyone successfully installed dfwiki (new wiki) from the http://appserv.lsi.upc.es/palangana/moodle/course/view.php?id=15 page?

It completely hosed my setup resulting in

Module "blocks" is not readable - check permissions

Module "course" is not readable - check permissions

Module "lang" is not readable - check permissions

Module "wiki" is not readable - check permissions

and an unusuable wiki. It apparently trifles with some other directories as well.

I followed their instructions, I guess I should have asked here first! smile

For some reason my backups are not restoring functionality, so I am off to re-install. Glad I don't have any students yet!

In reply to Nigel McNie

Re: Syntax Highlighting filter

by Satish Talim -
a. Instructions for enabling the plugin are still unclear.
b. Also, the exact code to use for the plugin to work is unclear too. Is it with html enabled or disabled.

I am using 1.8.2+

All help appreciated.
In reply to Satish Talim

Re: Syntax Highlighting filter

by Paul Preney -
I didn't test my edits with the HTML Editor disabled. It certainly works with it enabled. It should work with it disabled since the filter is enabled itself --unless the non-html mode messes that up.

I posted stuff for v1.9 Beta 3. Unfortunately, v1.9 Beta 3 was/us not ready yet, so, I went back to v1.8.4+.

With v1.8, more edits to get the filter to work are required. Moreover, I discovered that more edits are required to be done to the HTML Editor and various modules PHP files. The changes are required to prevent the HTML Editor from changing any of the whitespace and for the module from doing the same and/or to disable autolinking in pre sections, etc. In short, it is a bit of a mess --especially since the regular expressions that do the matching of things are not easily edited.

That said, I have things working. Essentially, I have edited the HTML Editor to not strip whitespace, etc. I have modified some key files for some modules (but not all of the standard ones). One reason I have not posted these edits is I am waiting for the v1.9 release. If the v1.9 Beta 3 was any indication, many fewer edits (i.e., the ones I posted were all that were required) to achieve success with the syntax highlighter.

I ultimately did use a "syntax" attribute within pre tags. This caused the least number of issues and works well. I also updated GeSHi to use the latest version and updated the script to autogenerate the CSS to the latest version. Due to the significant number of files to edit in v1.8, I thought it would be better to wait for v1.9.

Summary: IMHO it is not worth getting it to work with v1.8 and earlier (short of what already works) unless one is willing to debug/tweak module files and ensure that updates go smooth. (This is something I have done and I am doing.) With v1.9 it appears to be worthwhile, but, v1.9 is not ready for production sites.
In reply to Sergey Petkov

Re: Syntax Highlighting filter

by Miroslav Fikar -
I have the same experience. Line numbering is shown correctly, but formatting and coloring not. It was a simple bash code.
In reply to Miroslav Fikar

Re: Syntax Highlighting filter -- It WORKS!

by Paul Preney -
I've managed to get everything working INCLUDING with colour. I am running:
  • Moodle v1.6.1+ (CVS)
  • PHP5
  • HTML Editor enabled
  • Version of highlighter downloaded: the version that is current available via the Download Module page (i.e., I did not use anything posted to this page).

Colour syntax highlight at first did NOT appear. I investigated and it appears that if you attempt to add more stylesheets to the standard theme, it won't enable them. So, instead I created a new theme directory, copied all of the standard theme files into it, and then created the geshi.css file and edited config.php. After I switched the theme in the admin menu, the colour highlighting worked.

I am able to use the span tag as described in the README.txt file and full highlighting occurs. I will be doing more fancy tests later.

In reply to Paul Preney

Re: Syntax Highlighting filter -- It WORKS!

by Geoffrey Rowland -
Picture of Plugin developers

Paul

Thanks for this lucid description of how to get the syntax filter working in full colour. This is now working for our installation on Moodle 1.6.2, PHP 4.39. Initial tests indicate it is working well for php, html4strict, java and vbnet.

To maintain sanity surprise, I recommend turning off the HTML editor in your user profile, at least for the duration of editing code for the syntax filter. Otherwise it tends to strip out spaces, character returns etc, and generally stuff the formatting sad .

A big thanks to all those involved in the development of this filter cool.

Geoff

In reply to Geoffrey Rowland

Re: Syntax Highlighting filter -- It WORKS!

by Paul Preney -
You are all welcome!

It just bugged me like crazy that the line numbering worked, but, not the colour. Really it made no sense. Thus, I used the Firefox Extension "Web Developer Toolbar" (which is an excellent tool to examine CSS, etc.) and determined that the Geshi stylesheet was not being loaded after some edits to various Moodle files. I did this by hard-coding settings (and logging info. to the disk) and it showed that although the code was run, its output did not happen.

After seeing non-output of code, my gut said, "Well, what if this is not the STANDARD theme; --maybe to ensure that at least 1 theme works, the standard theme will not accept certain modifications?" So, I copied the directory over and made a "new" theme, applied that theme site-wide and reloaded the SAME page with Geshi-formatted code. Presto, it worked!!

NOTE: This "gut" feeling comes from years of programming experience. Don't ask me why I thought such; --it just did and such a talent (when it works!) really helps people like me find & resolve bugs/issues. smile

IMHO, it would be best to syntax highlight as follows (so HTML compatibility can be preserved):
  • Use the HTML code tag to enclose all code. This is what the code tag is for. IMHO span should not be used.
  • Use a CSS style type on the code tag to identify the type of code and the features required. There can be more than one CSS style within the style string which is where all of the options should be applied. In this way, there is nothing non-standard about the code.
Paul
In reply to Paul Preney

Re: Syntax Highlighting filter -- It WORKS!

by Nigel McNie -
A quick note: code tag is an inline element, and thus not suitable for large blocks of code (that's really what pre is for).
In reply to Nigel McNie

Re: Syntax Highlighting filter -- It WORKS!

by Paul Preney -
I'll grant that so long as traditional HTML is the vehicle. As long as it is such both code and pre are good candidates. smile

Notwithstanding the above for a least for some time into the future, looking into the future, XHTML- or XML-based sites shouldn't (and XML doesn't) rely on built-in rendering features of a tag. Clearly this should be specified (if nowhere else) with an appropriate CSS file that can override/set whether a tag (and/or specific class applied to that tag) is inline or block.

My comment was simply meant to address the earlier debate about which tag (e.g., span, etc.). If it is HTML pre or code is okay. If it is otherwise, then an aptly named tag would be better.
In reply to Paul Preney

Re: Syntax Highlighting filter -- It WORKS!

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
I was surprised that PRE was still in XHTML 1.0 strict, but it is, so it gets my vote.
In reply to Martin Dougiamas

Re: Syntax Highlighting filter -- It WORKS!

by Paul Preney -
With Moodle v1.7.1+ why does the HTML editor strip spaces out of the PRE tags? It is really annoying. When used to discuss source code (for a programming class), it is really nice to use the HTML editor and have code fragments in it marked as PRE. Clearly, C++ code needs to whitespace to remain readable, but, the editor strips them out unless done and submitted in raw mode --provided one NEVER switches back to WYSIWYG mode. If you re-edit the posting, then you must manually add all of the whitespace AGAIN within the PRE tags. Argh! IMHO the editor should not change anything inside PRE (or for that matter) CODE tags. Can this be easily fixed (whether I or someone else does it) so it is not a problem in future releases?? And if it is something I do, to whom & where would I submit the patch to? Thanks!
In reply to Paul Preney

Re: Syntax Highlighting filter -- It WORKS!

by Klaus Mueller -
Hi everybody
(maybe a stupid question)

I installed Syntax Highlighting for test purposes and it didn't work sad.
After looking in the source code I think the filter requires the 'old' <span ...> syntax and doesn' accept [code ...]
Is that correct?
Or does maybe the link for downloading the filter point to an old version?
Best regards
klaus
In reply to Klaus Mueller

Re: Syntax Highlighting filter -- It WORKS!

by Patrick Pollet -
Same problem here . Source code is still tagged
//$Id: filter.php,v 1.8 2005/05/19 17:32:30 defacer Exp $ in the latest downloadable version.

It do search for old span syntax, so does not work with HTMLEditor .

Attached a version that search the new [code ] markers . Only tested with Moodle 1.7 and php5

Cheers.




In reply to Nigel McNie

Re: Syntax Highlighting filter -- It WORKS!

by Stuart Anderson -
The filter works fine under Moodle 1.6.2, but doesn't install on my 1.8+. Things have gone very quite on the syntax highlighter front.

Nigel: Have now put to one side any further work on this filter? May be some one out there could take over development to get it working with 1.8? If I can get my PHP skills up-to-date and learn how moodle is stitched together, I may have a look myself. But if there is any one with strong PHP already out there, may be you could help out?!

It it were to work with the new nwiki module it would be great. The nwiki syntax has changed now so that links are created with my link as opposed to the old [my link] which means it doesn't conflict with the filter tags. As long as the wiki is set to work in nwiki format (i.e. without the terrible HTML editor), the formatting isn't screwed up and the filter should handle large blocks of code.

This is a lovely filter when it works and I would love to see it working again.

--Stuart.
In reply to Stuart Anderson

Syntax Highlighting filter -- doesn't work in 1.8+?

by Süreç Özcan -
Sorry, I don't know why my text come out so colorfull when posting...

I am using also Moodle version 1.8+ and would like to use the syntax highlighting.


In the readme.html there is stated:
To Install it:
- Enable if from "Administration/Filters".

I don't know how it worked on older versions, but if I go as admin to Admin->Modules->Filters, I can't find a filter called something like syntax highlighting or geshi to enable.


I have gone through the steps written by Paul Preney - Sunday, 27 August 2006, 11:36 AM way up on this page. So I have created a new theme with the geshi.css-file and so on...
I don't know how .css-files look usualy like, but mine looks like this and I am not sure if it is correct that it starts with ",":

-----------------------------
, 2005 * Based on work by Grigory Rubtsov , 2005 * * Uses GeSHi syntax highlighter 1.0.7.5 * http://qbnz.com/highlighter/ * @author Nigel McNie * * This program is free software; you can redistribute it and/or * modify it under the terms of the GNU General Public License * as published by the Free Software Foundation; either version 2 * of the License, or (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * http://www.gnu.org/copyleft/gpl.html **************************************************************/ // This script builds a (rather large!) but complete stylesheet // for using GeSHi in class mode. require_once("geshi/geshi.php"); $geshi =& new GeSHi('', ''); $output = '/** * GeSHi CSS file for Moodle * @author Nigel McNie * Released under the GNU GPL */'; $ignore = array( '.', '..', 'CVS', '.cvsignore', 'css-gen.cfg' ); $dh = opendir('geshi/geshi'); while (false !== ($file = readdir($dh))) { if (in_array($file, $ignore)) { continue; } $geshi->set_language(substr($file, 0, strpos($file, '.'))); $output .= preg_replace('#/\*\*.*?\*/#si', '', $geshi->get_stylesheet(false)); } $output .= "\n"; // Just to be silly $geshi->set_source($output); $geshi->set_language('css'); ?>
-----------------------------


To try it out I have created a new question type 'description' and tried out:
--------------------------
in WYSIWIG
--------------------------
public class Car{
private int milage;
}
---------------------------

--------------------------------------------------------------------------------------------------------------------------------
in HTML-Source
--------------------------------------------------------------------------------------------------------------------------------
<pre><span syntax="java">public class Car{
</span></pre><span syntax="java">private int milage; </span><br />
<div style="margin-left: 40px;"><span syntax="java"></span></div><pre><span syntax="java">} </span></pre>
---------------------------------------------------------------------------------------------------------------------------------
with and without the <pre>-tag and linenumbers="yes". As some people could see at least the linenumbers, I can't see any differences. Am I supposed to see the differences when changing to the WYSIWIG view or only after saving the question?

I hope some of you have expierenced a little and could give me one or another advice!

@Stuart: Can you use the highlighting by now? Does it not work only for nwiki or at all?

Thanks,
Süreç
In reply to Süreç Özcan

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Simon H -
Hi

I have a problem using Geshi in moodle 1.8.2+. When I write a source code in any language Geshi doesn't show me double quotes as we can see from the picture below.

hello

Instead of that I can see &quot;

Anybody know where is the problem? Single quote works fine. I'm using a Geshi for moodle which has been downloaded from http://geshi.org/downloads/moodle-geshi-filter-0.1.0.zip
In reply to Simon H

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Alexandr Borodin -
1. Open /filter/geshi/filter.php
2. Find the block beginning with comment
// Tidy up the source.
// Replace HTML entities which have been converted for display. GeSHi
// will reconvert them as it highlights, in the meantime they just get
// in the way.
3. Add to array of entities '&quot;'
4. Add to array of replacements '"' (double quote mark in single quote)
5. Have fun!
In reply to Süreç Özcan

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Süreç Özcan -
any further suggestions so far to my post from August 9th? I would really appreciate any help!
In reply to Süreç Özcan

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Paul Preney -
Since it is over one month later, I will answer your question as follows:
  • I have just upgraded my site from v1.8+ (CVS) to v1.9 Beta 3 (CVS).
  • I have installed the latest syntax highlighting filter from the Module page for such.
  • I REPLACED the filter.php (see the earlier post above) after unzipping the geshi.zip file into my */filter/ directory on my server. The reason is it uses square brackets instead of angle brackets allowing it to be used with the HTML editor --which is important for forums.

and I discovered that if one writes:

The following is some sample code:
[code syntax="cpp" linenumbers="yes" indentsize="2"]#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{
cout << "This is some text." << endl;
}
[/code]

Everything (almost) works. There appears to be an indentation issue with the HTML editor --I will examine and comment in a future post shortly.

Nevertheless to get the colours you need to also:

  • Open http://yourmoodledomain/moodledir/filter/geshi/styles.php
  • Copy-and-paste the result to a file called "geshi.css" your Moodle theme directory. If you don't have your own theme, then I highly recommend that you do. Simply:
    • cd moodledir/theme
    • mkdir mytheme
    • cd mytheme
    • cp -rv ../standardwhite/* .
      • Or replace standardwhite with the theme that you are using.
    • Edit config.php and add 'geshi' to the $THEME->sheets array.
      • e.g., see formal_white's config.php if you don't see an array.
    • Make sure that geshi.css is in the same directory.
    • Ensure that read permissions are set for the entire theme directory.
Also, please note that I put the Geshi filter at the TOP of all files on the Administration|Appearance|Modules|Filters page.

Hope this helps!

Paul
In reply to Paul Preney

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Süreç Özcan -
Sorry, like in my former long post there happen some formating which I didn't do which makes my post read difficult on some parts.

Moodle version v1.8.2+

Hi Paul,
sorry for such a late response. I have just seen your posts and am very happy to have found a communication partner!

I went through all these steps (the "Note:" text is where I am not sure or/and am having questions):
  1. Set up "Filter: Syntax Highlighting"
  • Download "Filter: Syntax Highlighting" from "Moduls and plugins" page: http://moodle.org/mod/data/view.php?d=13&rid=192 (Note: I have used my downloaded version from August 2007. Should things have been changed in the module so that I rather should download the current version?)
  • copy and paste the extracted folder "geshi" into the path moodle\filter\ (Note: double clicking into the extracted "geshi" folder should list another geshi-folder and the README.txt, filter.php and styles.php -> so far I had the geshi folder in another geshi folder and so as a result I couldn't find the filter to enable in admin-mode)
Enable Geshi-filter from "Administration/Filters" to use syntax-highlighting also in forums:
  • I REPLACED the filter.php (see the earlier post above: by Patrick Pollet - Sunday, 27 May 2007, 05:37 PM) after unzipping the geshi.zip file into my */filter/ directory on my server -> (moodle/filter/geshi/filter.php)
  • The reason is it uses square brackets instead of angle brackets allowing it to be used with the HTML editor --which is important for forums. (Note: I did this step although I just need the syntax highlighting in quizzes. Should I still leave the new filter.php in my directory or can I delete it? Why?)
Open http://yourmoodledomain/moodledir/filter/geshi/styles.php (e.g. http://localhost:8080/moodle/filter/geshi/styles.php) Copy-and-paste the result to a file called "geshi.css" in your Moodle theme directory. (theme\(your new theme copy)\geshi.css)
  • If you don't have your own theme, then I highly recommend that you do. Simply:
    • (copy the standardwhite theme folder and rename it to your own OR DO IT USING FOLLOWING LINES) (Note: I just copied the 'standardlogo' theme folder and renamed it to 'standardlogohighlighting'. Do I need to copy standardwhite-theme instead?)
    • cd moodledir/theme
    • mkdir mytheme
    • cd mytheme
    • cp -rv ../standardwhite/* . (Or replace standardwhite with the theme that you are using.)
Edit config.php and add 'geshi' to the $THEME->sheets array.
  • e.g., see formal_white's config.php if you don't see an array.
  • to find in folder: theme\(your new theme copy)\config.php -> $THEME->sheets = array('gradients','geshi');
Ensure that read permissions are set for the entire theme directory. (Note: Since I use it locally at the moment I don't have to take care of this - right?)
Also, please note that I put the Geshi filter at the TOP of all files on the Administration|Appearance|Modules|Filters page. (Note: I guess you mean Administration/Modules/Filters page? If not, where to find?) Now I have opened a new description quiz type to try it out with the example code from you above and copied the following into the HTML-editor (NOT HTML-Source!):
  • [code syntax="cpp" linenumbers="yes" indentsize="2"]#include <iostream>

    using namespace std;

    int main(int argc, char *argv[])
    {
    cout << "This is some text." << endl;
    }
    [/code]
  • Note: when saving the question I see line numbering but no syntax highlighting -> see attachment!
  • Note: Do I have to type anything else like pre-tags? Could you give me an exact example of what should be included in the HTML-editor and HTML-Source-editor, please?
So far I didn't go through the next steps of your later posts in order to get the tabbing correct. I want to go through them when I can see syntax highlighting first.
I didn't get the whole idea yet - If I want to write a source code as a description, do I only type my source code into the HTML-editor and after saving I see syntax highlighting? Can't I see the highlighting before saving my question type? I am asking because I could see the line numbering only after saving the question, but it would be nice having syntax highlighting while typing. Should I be able to copy source code from another editor into the html editor and getting syntax highlighting?
What I want to do is expanding Moodle by allowing to create a question type called "source code" adding java code instead of a normal description with syntax highlighting. Other features of my new question type shall be:
  1. Several editors: Each editor should include one java-class. If more classes shall be added to the "source code"-question type then this should be enabled through an "add new class"-button which displays a new editor. (Note: using the html-editor needs some time for uploading - therefore it might not be smart using it?)
  2. Source code editor arrangement: There shall be a select-option to display one or two source codes per row. This way longer source codes get a readable display on the monitor. (so far I had used the table option in the html-editor which didn't work all the times - I don't know why.) (Note: I don't know if this is possible to realise in Moodle at all though.)
  3. Next to it I would like to enable for each displayed source-code-editor uploading a .java-file from e.g. the users desktop into the questions (html-?)editor, downloading the current code into a .java-file on e.g. the users desktop and deleting the editor in case it won't be used anymore.
  4. Requirement: It would be nice to be able to fill up an editor by whether:
  • uploading a .java-file or
  • through own input while creating this question type or
  • through a copy and paste from another file into the editor.
So what interests me is if someone could tell if I should use the HTML-editor or if I should use another syntax-highlighting supporting editor which is not as complicate as the html-editor? I have no clue of how complicate it would be to enable such a feature in moodle. Therefore my first idea was to use the html-editor instead (only if I first get the syntax-highlighting set though wink).

Any ideas?
Attachment cpp_highlighting.JPG
In reply to Süreç Özcan

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Paul Preney -
NOTE: My posting and I presume the parent post's too formatting is broken. Parts of my post don't look like what they do in the HTML Editor.

Okay, if you are seeing line numbers, then you are in business --the filter is working. To see colours, you must load:
  • http://yoursite/pathtomoodle/filters/geshi/styles.php
and save the CSS output as geshi.css in your theme/YOURCUSTOMTHEMEDIR/. Don't forget to set the proper file permissions. smile

Make sure that theme/YOURCUSTOMTHEMEDIR/config/.php has 'geshi' in the following line:
  • $THEME->sheets = array('styles_layout', 'styles_fonts', 'styles_color', 'geshi');
You may need to purge your browser's cache for things to reload properly, but, everything should work.

To enable the GeSHi filter for Java, use syntax="java" instead of syntax="cpp".

I like your idea of a specific editor configured for programming code. That might be the easiest way to make code work. However, it would require some modules to allow "multiple" information types --like the question module. So, for general use across modules more thought is required. Your idea ought to work very well for a programming question type, however.

Currently, Moodle supports only one editor AFAIK, however, from looking at code in v1.9 it looks like it is being expanded to support more than one. The last I checked (early Jan.) the documentation on this new ability is very poor. Hopefully, it has improved, but, I don't know how flexible it is. Ideally, I'd like to see what you suggest (i.e., a programming-specific editor) and the ability for the HTML-type editor to support inline code with it easy to specify which one to use.

I am very busy today, so I will completely respond to your posting later this week. smile
In reply to Paul Preney

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Süreç Özcan -
What I have listed in my former post above includes a list of all the steps I did including the steps you have suggested me above:
  • saving the css output as geshi.css (I had forgotten to comment out the first two lines above the generated code, but it didn't make any changes in syntax highlighting)
  • I checked the permissions of the moodle\theme-folder through right-click -> properties -> General-tab -> Attributes: Read-only, which is already checked -> check again -> Apply -> Apply changes to this folder, subfolders and files -> ok -> ok That should do what you ask me to do, right? -> this didn't make any difference in syntax highlighting though. Do I have to set these permissions to the theme-folder (as I did) or only my newThemeFolderDir?
  • purging browsers cache: I am using Firefox and what I did is going to: Tools -> Options -> Advanced -> Network -> Cache: Clear Now => no difference in syntax highlighting
I would be very thankfull if you could read more carefully over my steps I went through in my former post and may realize sth. I forgot!? (e.g. the note about copying standarlogo instead of standardwhite, because my institution I am studying at needed a logo included)


Now to the other interesting part:
Actually the idea goes further.
I would like to separate the given source code (included by the "source code" question type) from the related questions. Therefore after I got this new question type "source code" working I would like to expand some question types (probably Multiple Choice, True/False, Short Answer and two others) by adding an extra part into the creating form above the "Question text"-part. For this I would like to include a button group and below it the Source-Code (not directly editable but through the buttons)
from an created "source code"-question type.

In my conceptual design I have planned the following for the 3 buttons:
  • new: shall open a new window with the empty "source code" question type user interface in order to creating a new one, saving and displaying it in the question.
  • reuse: (maybe a select instead of button) shall list all so far created "source code"-question titles from which to select and display one in the current question-type.
  • change: opens the proper "source code" question type that had been displayed already in order to change and save and redisplay.
This way a teacher can see the source code in connection/combination to the currently creating new question(-type). Note that the source code won't be redisplayed in each question - it is more a help while creating a new question to the corresponding source code.

Later in a quiz there can be set "page breaks" whenever there starts a new "source code" question type with its corresponding questions. Do you get my point?

So do I get you right that you suggest me using another editor and not the HTML-editor for my purposes?
You mention some problems about multiple information types for general use across modules. Would I run directly into them with my bigger plan or do they stay out of reach since I am moving only inside of the question-folder?
We have 1.8.2+ installed at our department. That is why I am developing locally at the same version at the moment. Developing in v1.9 wouldn't make any sence then - what do you say?
I don't get your sentence that starts with "Ideally, I'd like to see..." at the end of your post. Could you describe it in other words, please!
In reply to Süreç Özcan

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Süreç Özcan -
After I went through each and single step over and over I have realized my "slip of the pen":
Through all my trials I forgot to reselect my new created theme in the Admin->Appearance->Themese->Theme Selector.

Now syntax highlighting works - Yipieeeeee big grin
In reply to Paul Preney

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Süreç Özcan -
Maybe I should open a new discussion topic in the "Quiz module"-forum for my plans of increments. What do you say?
This way I don't fill up the syntax-highlighting forum with my stuff and other people may have a look at it as well.
In reply to Süreç Özcan

Re: Syntax Highlighting filter -- doesn't work in 1.8+?

by Paul Preney -
Okay, some more tidbits:
  • Even with <pre>...</pre> syntax, while <pre> tags should not have whitespace stripped, the htmlarea seems to do this anyways. Sigh. (I am still looking at this.)

  • I have traced the reason for whitespace not being preserved by the HTML editor to:
    • lib/editor/htmlarea/plugins/GetHtml/get-html.js

  • In get-html.js, one really would prefer having as few edits as possible --preferably none. Alas, for syntax highlighting and other editor types, one WILL have to edit such.

  • With the syntax editor, it is a really, really GOOD idea to remove all smiley translations from the HTML Editor administrator page. While some users might like smileys, they cause problems with source code and other punctuation that can be used.
So what does this all mean?
  • If we use <pre></pre> (i.e., normal HTML-style tags) and you want the user to type in code using the HTML editor:
    • The editor will escape < with &lt; and > with &gt;.

    • The escaping will cause the HTML editor, which should normally not process content inside of <pre>...</pre> and <script>...</script> tags to miss "skipping" such. This causes whitespace to be removed. sad

    • There is a problem with using <pre> in that one has to enter the "raw HTML" mode to edit the syntax-highlighted block. This is not fun --even for coders.
      • NOTE: The filter does work if you are willing to do this.
      • NOTE: Some editors like TinyMCE allow a pull-down according to a stylesheet so that it can add a value to the class attribute (e.g., class="codehighlight"). Unfortunately, it doesn't appear that the htmlarea supports this --further argument to switch from htmlarea to a different HTML editor.
      • NOTE: Apparently the issue with TinyMCE is that its file manager/uploader is not free. This is needed for images, etc. and so others are suggesting that TinyFCK=TinyMCE+FCKeditor be used instead.
      • NOTE: Also, apparently, htmlarea is no longer under development.

  • To avoid the previous issue, one might think we could use something like [pre]...[/pre] if the editor's "skipping" syntax is fixed to address this.
    • This is okay, except this syntax is used by the various wiki formats --including efwiki (which is the current default wiki in Moodle).

    • Not using < and > means that we can write code WITHOUT viewing raw HTML --which is not fun especially in long posts. This does mean that the HTML editors will need a small tweak so they leave whitespace intact for such syntax.
      • This is not the way it should be done however --we should use tags so it is XML-compliant. Using namespaces with those tags can allow proper processing and validation of all filtered content.

    • NOTE: If you allow <pre>...</pre> to be type (and therefore < and > are escaped), then you need to convert the escaped <pre>...</pre> when entering and leaving raw HTML mode in addition to when one uploads the content. The problem in using a standard tag like <pre> is that it will convert ALL <pre> tags this way --which is not desired-- and one does not want to change the editors too much.
      • The only way around this is if an editor allows one to enter special "not understood" tags that it shows "raw" (including embedded content) to the user. This might be the best solution since we could use an XML namespace to define such (e.g., <mf:code>...</mf:code>) which is what should be done. While partial validation can be done by the editor, it should be simple --the server can do the real work. If it is XML-conforming, then it would be easy to validate the resulting submission --with namespaces. In my example, the "mf" in "mf:code" could simply refer to the "Moodle filter" namespace.
All-in-all this really shows that Moodle needs its content handling to be streamlined (i.e., properly use XML, XSLT, XML validation, and XML namespaces) so that it is clean and well-formed especially when invoked multiple times. That said, the primary problem is not the syntax-highlighting filter or all inside of Moodle itself. Instead, the problem is really the Moodle editors + their configuration + the mess of Moodle's content handling all together.

I've not submitted this as a bug report since it needs discussion. Even if you rant, please give constructive suggestions, etc. on how:
  1. We can make this filter work.
  2. We can improve Moodle to get rid of this mess.
Sincerely,

Paul
In reply to Süreç Özcan

Fixed --for now! :-)

by Paul Preney -
Okay, here are some acceptable short-term fixes to get things to work with the HTML editor. This fix requires edits as follows [as found in v1.9 Beta 3 CVS as of Dec. 28 (I did a CVS update just before my first post for Dec. 28 on this site)]:
  • lib/editor/htmlarea/plugins/GetHtml/get-html.js on line 102:
    • $ diff get-html.js get-html.js.orig
      102c102
      < string = $3.replace(/[\n\r]/gi, " ")./*replace(/\s+/gi," ").*/replace(c[14], function(str) {
      ---
      > string = $3.replace(/[\n\r]/gi, " ").replace(/\s+/gi," ").replace(c[14], function(str) {

  • lib/editor/htmlarea on line 2775:
    • $ diff htmlarea.php htmlarea.php.orig
      2775c2775
      < string = $3.replace(/[\n\r]/gi, " ")./*replace(/\s+/gi," ").*/replace(c[0], function(str) {
      ---
      > string = $3.replace(/[\n\r]/gi, " ").replace(/\s+/gi," ").replace(c[0], function(str) {
Notice that the replace(/\s+/gi," "). text is commented out. This is the editor code that is guilty of removing whitespace from the <pre> tag. Doing this allows one to use the <> button and it will not change your formatting anymore! smile

Now, this leaves filter/geshi/filter.php:
  • Download the syntax highlighter filter from the Moodle modules page.
  • Install the filter. Don't replace the filter.php with the earlier version on this page. Instead, edit filter.php on line 51 as follows:
    • $ diff filter.php filter.php.orig
      51c51
      < $search = '/<pre (.*?)>(.*?)<\/pre>\s*/is';
      ---
      > $search = '/<span (.*?)>(.*?)<\/span>\s*/is';

    • i.e., change "span" to "pre".
Now, you can use the filter (assuming you've enabled it). If you already had the filter enabled, then you may need to shift-reload the relevant pages (including shift-reloading the pages with the editors to pick up the .js file changes, etc.) --it might be easier to purge your browser's cache.

To use the filter, do the following:
  • Invoke something that uses the HTML editor.
  • Create a Preformatted section. Type in some text.
  • Click on the <> button to edit raw HTML.
  • Add the attributes you want to the pre tag, e.g.,
    • indentsize="2" linenumbers="yes" syntax="cpp"
  • Click on the <> button to return to normal mode.
  • Continue adding/editing your code/text.
  • Post it.

You'll notice that indentation is preserved and you can re-edit the code without clicking on <> since the attributes are preserved as well. Thus, only once per <pre>-code-section will you have to add the required filter attributes.

For those that don't like to read code, the syntax attribute allows the following types:
  • actionscript, ada, apache, asm, asp, bash, c, c_mac, caddcl, cadlisp, cpp, csharp, css, delphi, html4strict, java, javascript, lisp, lua, mpasm, nsis, objc, oobas, oracle8, pascal, perl, php, python, qbasic, smarty, vb, vbnet, visualfoxpro, xml
and indentsize will only replace tabs with spaces --it will not adjust spaces.

I think, until Moodle gets an XML content-handling engine going, this is satisfactory and it won't break with upgrades assuming:

  • A request (i.e., bug report) is entered to fix the two htmlarea files.
  • A request (preferably the same one) is entered to fix any required tinymce files.
    • Would someone test this please? I am not using tinymce and I haven't been able to figure out how to make it default to such.
  • The syntax highlighter filter is amended to use <pre> and NOT <span> tags.
Sincerely,

Paul

In reply to Paul Preney

Re: Fixed --for now! :-)

by Paul Preney -
I submitted the HTML editor bug portion of this in the Moodle Tracker here:
As for the syntax highlighting filter, the maintainer of such hopefully will make the change in filter.php from "span" to "pre".

I doubt that much more can be done on a practical level for now --I tried to make the minimum number of changes in the proper locations. Given only 3 trivial edits total to pre-existing files, I think I was successful with such. Later, when Moodle is doing XML stuff, a better version of this filter and how it interacts with the editor(s) and Moodle can be implemented.

Again, let me know your thoughts, etc. and I am open for those wishing to debate such, or, future improvements.
In reply to Paul Preney

Re: Fixed --for now! :-)

by Süreç Özcan -
Hi Paul,

how great... it works (partly)!!! smile
I am not sure if the tabbing works as it should. Could you have a look at this, please?
  • I have removed "replace(/\s+/gi," ")." from both files you mention.
  • I didn't install the syntax highlighting filter again so that I didn't install the filter for a second time as well. Should I?
  • Editing filter.php: In my code there is written:
******************************************
function geshi_filter($courseid, $text) {
// $search = '/<span (.*?)>(.*?)<\/span>\s*/is';
// test PP
$search = '/\[code (.*?)\](.*?)\[\/code\]\s*/is';
return preg_replace_callback($search, 'geshi_filter_callback', $text);
}
******************************************
So I don't know why to change "span" into "pre" if it won't be used anyways?!? I left it like it is and tried out copy and pasting a java code into the HTML-editor:
******************************************
[code syntax="java" linenumbers="yes" indentsize="2"]
public class Factorial {

public static double factorial(int i) {
if (i==0)
return 1.0;
return i * factorial(i-1);
}

}
[/code]
******************************************
Saving the question and looking at it in the preview shows the source code with tabs! smile
Going back to edit the question displays the source code without tabs though. mixed Is this meant to be like this at the moment?

Thanks ahead,
Süreç
In reply to Nigel McNie

Creating a Theme

by Arunodaya Yeshep Ashade -

Dear All,

How do I create a Stylesheet in Moodle to create our own theme?

Please help & suggest,

Regards,

Arunodaya

In reply to Arunodaya Yeshep Ashade

Re: Creating a Theme

by Süreç Özcan -
If you want an own theme in order to get syntax-highlighting work correctly... for me it worked just to copy an existing theme e.g. "standardlogo" and renaming it to "standardlogohighlighting".
In reply to Nigel McNie

Re: Syntax Highlighting filter: automatic linking?

by Süreç Özcan -
I am using the syntax-highlighting-filter for java-code and have realized that the word "System" in the line "System.out.println();" gets automatically linked when displaying the syntax-highlighted source code (or other keywords like "String", "Exception", "NullPointerException", "StackOverflowError").
Is this a bug?

In Admin->Modules->Filters->Manage filters I have only Geshi enabled.

Trying to use the "Prevent automatic linking"-button in the HTML-editor displays:
"<span class="nolink">System</span>" in between the synstax-highlighted source code.

Does anybody know what I could do or will I have to accept this when using this filter?
In reply to Nigel McNie

Re: Syntax Highlighting filter

by Paul Craven -
This doesn't seem to work in the current Moodle version for us. All the code is displayed on one line with no linebreaks.
In reply to Nigel McNie

Ynt: Syntax Highlighting filter

by burak inner -
I installed the filter without any problem..
I can use the filter..

I download http://geshi.org/downloads/moodle-geshi-filter-0.2.0.zip and then download the latest version of geshi from http://sourceforge.net/project/showfiles.php?group_id=114997 here..

and then I replaced the moodle-geshi-filter-0.2.0.zip file's geshi folder with the latest version of geshi..
and then upload to the server..(this is only update geshi to 1.0.8.6, doesnt effect geshi filter)


I can use the syntax highligting filter with the pre tag..
but I put some html code in [code] tag.

everything is perfect for the first time.. I can see the html code with syntax highlighting but when I try to re-edit the page with the WYSIWYG editor every thing is messup.

so something really needs for the WYSIWYG editor..because the problem is the WYSWYG editor.

I want to test this problem with a locally installed Moodle 2.0 version .. I hope the new WYSWYG editor in Moodle 2.0 doing great with the syntax highlighter...
Average of ratings: Useful (1)
In reply to burak inner

Ynt: Syntax Highlighting filter

by burak inner -
new WYSWYG editor in Moodle 2.0 has the same problem with geshi..
I test it on a locally installed Moodle 2.0

In reply to burak inner

Re: Ynt: Syntax Highlighting filter

by Sam Mudle -

everything is perfect for the first time.. I can see the html code with syntax highlighting but when I try to re-edit the page with the WYSIWYG editor every thing is messup.

so something really needs for the WYSIWYG editor..because the problem is the WYSWYG editor.

Once you edit the Geshi code, the Moodle editor will "convert" all of your code to it's HTML parser.

Yeah, I wish Moodle would dump it's WYSIWG editor and use one of the MUCH better open source editors out there.  TinyMCE has fantastic Geshi compatibility.thoughtful

In reply to Sam Mudle

Re: Ynt: Syntax Highlighting filter

by Sam Mudle -
I will add that if you compose a TEXT page resource instead of an HTML page, then it's much easier to edit.
In reply to Sam Mudle

Re: Ynt: Syntax Highlighting filter

by hans paule -

i dont know what is the right topic ? This or:

http://moodle.org/mod/forum/discuss.php?d=25413

?

 

What is the current version of geshi for moodle 2.x ?

My output over [code php] ... [/code] is more than shit.

i have uploaded the content from moodle-geshi-filter-0.1.0.zip

in my filter dictionary and have aktivate it. But the result is very bad.

The editor under the tinymce field is "HTML format" is it right ?

Or musst i write in the  HTML Source edit ?

Sorry for my bad english.

LG

 

My output is like this:

</p>
<p><?php</p>
<p>$eingabe = "text";</p>
<p>function();</p>
<p>echo ("Das ist eine eingabe");</p>
<p>?></p>
<p>
In reply to Sam Mudle

Re: Ynt: Syntax Highlighting filter

by Les Bell -

Is anybody using the GeSHI filter successfully with Moodle 2.9?

I've installed the filter - from the filter_geshi_moodle26_2014040800.zip file, which appears to be the latest version referenced in the various discussion threads - and have enabled it. I've tried to insert some Python code into a wiki page, just the first of many such insertions for a major module on programming languages. I've tried it using [code python][/code], I've tried it using <pre><span syntax="python"></span></pre>. I've tried it with the Atto editor; I've tried it with the TinyMCE editor; I've tried it using a plain text area. I've tried it while wearing a lampshade on my head and reciting Gray's Elegy in a Country Churchyard. I was hoping an animal sacrifice might help, but my wife grabbed the dog and ran screaming from the room.

Nothing works.

At best, the <pre></pre> tags survive and I get some double-spaced (!?!) code in a grey box. However, the <span syntax="python" linenumbers="yes"> tag always gets reduced to <span>.

Does anyone have the required incantation, or does the dog get it?

In reply to Les Bell

Re: Ynt: Syntax Highlighting filter

by AL Rachels -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I think the dog is safe. I tried it at first in the front page News forum of my site, and it kept failing. I finally realized that the example pictures were of code typed into a course summary, so I tried it there and in a topic and it worked in both places. Since then I've tried it in a book description and a book page...worked in both. So it might depend on where you are trying to use it. Tried all those places at first with php since that is what they used in the examples. I then changed to python and it also worked.

Using latest GeSHi filter and Moodle 2.9.1+ with the Atto Editor. I also used &lt; and &gt; where necessary if I went back in and edited anything. Works without the linenumbers="yes" too.


In reply to AL Rachels

Re: Ynt: Syntax Highlighting filter

by Gareth J Barnard -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers

I've successfully implemented Alex Gorbatchev's Syntax Highlighter - http://alexgorbatchev.com/SyntaxHighlighter/ - in my Shoehorn theme: https://moodle.org/plugins/view.php?plugin=theme_shoehorn.  See a demo here: 

In reply to Gareth J Barnard

Re: Ynt: Syntax Highlighting filter

by Les Bell -

Thanks for that, Gareth - I hadn't considered using a client-side syntax highlighter, but that looks like a very useful alternative, particularly for parts of Moodle where the GeSHI filter doesn't work. I also have to get syntax highlighting, or at least code formatting, working on the Joomla part of my site, and this approach might kill two birds with one stone.

--- Les

In reply to AL Rachels

Re: Ynt: Syntax Highlighting filter

by Les Bell -
You're a scholar and a gentleman, AL Rachels - even if you are covered in dog slobber (the dog is very grateful!).

You've identified the problem - I was trying to use GeSHI in a wiki (my lecture slides actually link to related articles in the wiki). So I created a book - which will work for this part of the course, anyway - and it works!

For the benefit of anyone else following this, I found that:

  • The technique of using [code python][/code], formatting it as "Pre-formatted", then switching to HTML view to paste the code fragment in doesn't work - it just displays as preformatted, with the "[code python]" and "[/code]" tags visible and not interpreted.
  • Using "<pre><span syntax="python"></span></pre> just gives double-spaced preformatted code with no syntax highlighting, and
  • Editing the HTML to insert <span syntax="python"></span> works beautifully - there's a veritable rainbow of syntax highlighting on the screen. ;)
So now we know the GeSHI filter works in books, but doesn't work in a wiki. I expect it works in pages and lessons, too, but not forums. It will be interesting to see where else it works - or doesn't. sad

Thanks for your help with this!

--- Les
In reply to AL Rachels

Re: Ynt: Syntax Highlighting filter

by Maya Lekova -

Anyone had luck installing the GeSHi plugin on Moodle 3.0? Thank you in advance!

In reply to Maya Lekova

Re: Ynt: Syntax Highlighting filter

by Maya Lekova -

Actually just tried it and it seems to be working fine in lessons smile