I've written a text filtering system in Moodle and added it to CVS.
This means the admin can set up one or more functions to be applied to all HTML texts that are output by Moodle (eg section summaries, forum postings, text and HTML resources, etc ...)
(Similar functionality was available for the Library add-on, but this is a new standard system that can be applied to multiple filters).
The first working example is automatic links from Will's glossary module. There is not currently a GUI to set this up - you need to add this line to your config.php:
$CFG->textfilter1 = 'mod/glossary/dynalink.php';
Note you can also have textfilter2, textfilter3 etc ... and they will be applied in that order.
Each file pointed to must contain two things:
- a variable $textfilter_function with the name of a function
- the function itself, with parameters ($courseid, $text), returning the filtered text
The caveat is that the filters must be able to deal with HTML ... if they add links they should NOT destroy existing links, for example.
Some ideas to get your imagination going include: the multilanguage filter, an English -> Slang filter, a 'multi-coloured letters on holidays' filter, a "remove bad words" filter ... etc
Once we get some coming in I'll start a "filter" subdirectory in the distribution to start collecting them.
I've written a little censorship filter that provides a nice simple example of how to create a text filter. All it does is obscure those rude words - someone asked for it a while back.
You can find it in filter/censor/censor.php.
You can find it in filter/censor/censor.php.
Martin, want me to create a glossary of a lot more "bad" words for you? Your list is pretty short! Tempers must be more even down under. 
F*** no! Let those
w****** who need to f***ing censor other f***ing c***s do the f***ing
work themselves! 
Instead of filtering, you could teach your students how to behave in open society. Throw away the filters? No, you could use them as a signalsystem: more then two/three f**k words in a message and there goes an automatic warning to the student: and his f**k counter goes one up. When the counter reaches levelcount X, there opens an automatic dialogue between the student and his teacher, with the last text as starting point. O jey, there also goes an automatic copy to his parents ... and a copy to the local policestation.. and...and..
There is a joke of a Catholic priest and a Protestant minister going for a hunt.
When they see a rabbit, the priest takes his gun and shoot. "Sh*t, I missed him", cries the priest.
"Look out, you evoke the fury of GOD", warns the frightened minister him.
Then Again a rabbit and the priest misses again. "HOLY SH*T" and yes, there it happens...
In his anger GOD sends a thunderbolt....
...and the minister falls death on the ground and a thundering voice from above shouts:
"Sh*t, Sh*t, Sh*t!!!,.. I missed that bl**dy little sw**ring priest again!"
But the next generation ban-list that will grow is more disquieting: Darwin, evolution...
Some people will need loooong lists ...:
http://edition.cnn.com/2003/EDUCATION/05/28/life.language.reut/index.html
http://edition.cnn.com/2003/EDUCATION/05/28/life.language.reut/index.html
Suddenly I miss my childhood. Tomorrow I tell my son some jokes and teach him how to cuss the way we did back in the day...
Please John, just one, please..
By the way Martin, can't you approach the problem form another side: implement a spelling checking library routine, allowing in mesaages only the approved words..
(And also helping me with my horrible typing errors?)
Yes, good idea, only words in the glossary will be allowed.
Text filters are currently used in the following places:
- site summary
- category summary
- section summary
- user description
- label resource
- login instructions
- assignment description
- forum posts
- glossary introduction, definitions, comments
- journal introduction, entries
- resources of type HTML page and Wiki text
- workshop description
and maybe others that I am not aware of. Natural places where text filters should be used but aren't:
- course summaries
- activity descriptions on the index.php pages of modules
In general I think that any text that the user inputs in a multi-line textbox should be output with textfilters. Do you agree? Martin: do I have permission to include calls to format_text() in relevant places in CVS?
Absolutely, please do. Any missing ones are simply oversights.
Currently text filters are called after applying text_to_html to the text. So they really are html filters. Some filters however need to be applied before the text_to_html filter because they act on non-HTML text. For example TeX is a non-XML standard. It's been around since the sixties, after all. Any replacement of symbols inside a TeX equation will break it immediately. So a TeX filter certainly has to run before any other filter.
I propose to have a universal filter as part of format_text that passes everything inside start tag - end tag pairs through a registered filter.
One way to implement this would be to have a table mdl_filters with fields 'id', 'start tag', 'end tag', 'filter', where 'filter' somehow specifies the filter to be applied (for example by giving the name of the subdirectory of the filter directory where it can be found).
The current mechanism for calling textfilters would then only be used for filters that aren't triggered by tags, like the glossary filter for example.
This would by no means be a panacea for dealing with TeX. Most of the mangling of TeX occurs at the stage when text is entered into Moodle, say when editting a new forum post. For example, when you enter the inequality symbol < in raw html mode, it looks to Moodle like you are opening an HTML tag. Since whatever follows is very unlikely to be on the list of allowed tags, Moodle will strip away the < plus whatever text follows, and this will happen when the user clicks on "Save Changes". To avoid this, the < has to be encoded before this stage.
I think that to deal with this, we need better coordination with HTML Editor. This has already been discussed here. Whatever TeX survives the editting process, no matter how encoded, can always be easily restored to usable form by the text filter, no matter at which stage of text display it takes place.
I can't really see the problem you're trying to solve here ...
Most text entry will be in the HTML editor, so there is no conversion happening before the TeX filter (if you put the TeX filter first in your list). Well OK, there is smileys but I don't think these feature much within TeX code.
Text in non-HTML, such as Moodle auto-format does not change much when being converted to HTML. It gets a few line breaks and URLs get linked .. but I still don't think any TeX would get mangled ...
Most text entry will be in the HTML editor, so there is no conversion happening before the TeX filter (if you put the TeX filter first in your list). Well OK, there is smileys but I don't think these feature much within TeX code.
Text in non-HTML, such as Moodle auto-format does not change much when being converted to HTML. It gets a few line breaks and URLs get linked .. but I still don't think any TeX would get mangled ...
Not sure whether anyone will use this, but I'm using a copy of the censor.php file to directly link certain words to another website. This is used to direct to three sites that I refer to quite often.
I was manually typing the link each time, eg. <a href="http://thescotsman.co.uk/" target="blank">The Scotsman</a>
I know I could have created a Glossary entry, but in this one particular case, I preferred not to - it got a bit monotonous to keep on seeing a description of the site with a link at the bottom of the glossary entry.
So I changed the censor words in Martin's filter to allow links -see below for an example...
function correct_filter($courseid, $text) {
static $search = array('The Scotsman');
static $replace = array('<a href="http://thescotsman.co.uk/" target="blank">The Scotsman</a>');
return str_ireplace($search, $replace, $text);
}
In the real file, I only use three links.
I was manually typing the link each time, eg. <a href="http://thescotsman.co.uk/" target="blank">The Scotsman</a>
I know I could have created a Glossary entry, but in this one particular case, I preferred not to - it got a bit monotonous to keep on seeing a description of the site with a link at the bottom of the glossary entry.
So I changed the censor words in Martin's filter to allow links -see below for an example...
function correct_filter($courseid, $text) {
static $search = array('The Scotsman');
static $replace = array('<a href="http://thescotsman.co.uk/" target="blank">The Scotsman</a>');
return str_ireplace($search, $replace, $text);
}
In the real file, I only use three links.
I'm using something very similar on this site to make the word FAQ into a link.
By the way, I've now created a proper GUI for adding/managing text filters.
Admin -> Configuration -> Filters
By the way, I've now created a proper GUI for adding/managing text filters.
Admin -> Configuration -> Filters
The GUI sounds good.
I'm using Moodle 1.2 alpha (2004022100), but couldn't see the Filters option in the Admin -> Config screen.
Having said that, there is a filters option in Admin -> Config -> Variables. Is that what you meant?
ps. Does that mean that the Text Filters settings in config.php can be scrapped?
I'm using Moodle 1.2 alpha (2004022100), but couldn't see the Filters option in the Admin -> Config screen.
Having said that, there is a filters option in Admin -> Config -> Variables. Is that what you meant?
ps. Does that mean that the Text Filters settings in config.php can be scrapped?
It's only a few hours old, so it might take a while to get out to anonymous CVS ... the moodle-latest download will have it though.
Yes, the filter settings in config.php can be scrapped.
Yes, the filter settings in config.php can be scrapped.
This is truly excellent! Thanks Martin.
Martin, when I updated from CVS a few minutes ago, the filter GUI was present, but only the 6 'standard' Moodle filters show up. The two filter files that I added, one for links and one for spelling correction, are not visible. Do they need to be in a special place for the admin interface to find them? They are currently in filter/link/link.php and filter/correct/correct.php.
The actual filter file now needs to be called "filter.php" within each directory.
Filters are searched in all "mod/xxxxxx" directories and all "filter/xxxxxx" directories.
Filters are searched in all "mod/xxxxxx" directories and all "filter/xxxxxx" directories.
Clash of the Filters...
A word of warning with this modified text filter ... if, by chance, you use a phrase which is also in the Glossary, then the Glossary link does not work. What I saw (when I forgot that I had left an entry for 'Scotland on Sunday' in the Glossary) was...
Scotland on Sunday" href="http://development.baillie.org.uk/mod/glossary/showentry.php?courseid=2&concept=Scotland+on+Sunday" onClick="return openpopup('/mod/glossary/showentry.php?courseid=2\&concept=Scotland+on+Sunday', 'entry', 'menubar=0,location=0,scrollbars,resizable,width=600,height=450', 0);">Scotland on Sunday
although the last three words (Scotland on Sunday) were hyperlinked (I can't replicate that in this post).
A word of warning with this modified text filter ... if, by chance, you use a phrase which is also in the Glossary, then the Glossary link does not work. What I saw (when I forgot that I had left an entry for 'Scotland on Sunday' in the Glossary) was...
Scotland on Sunday" href="http://development.baillie.org.uk/mod/glossary/showentry.php?courseid=2&concept=Scotland+on+Sunday" onClick="return openpopup('/mod/glossary/showentry.php?courseid=2\&concept=Scotland+on+Sunday', 'entry', 'menubar=0,location=0,scrollbars,resizable,width=600,height=450', 0);">Scotland on Sunday
although the last three words (Scotland on Sunday) were hyperlinked (I can't replicate that in this post).
Actually, this isn't just a clash with the Glossary. The same thing happens with any duplication ... for example, I had one filter for 'Anite' and another for 'Anite Public Sector'. The second filter gave up trying!