Moodle Selenium Test

Moodle Selenium Test

by Christian Lebe -
Number of replies: 9
Hi all,

Here in Moodle HQ, we have been looking alternatives on automated testing Moodle for a while. And thanks to David Scotson, who has mentioned Selenium.

I have written some test cases in Selenium and would like to hear some feedback from the community. What I found is the test case is not really flexible. We need some time to sit and create a test environment, and this environment should be exactly the same each time we want to run the test case (for example link in quiz questions contains numerical id that should be the same for each test) otherwise the test will fail.

Due to this reason (the sensitivity reason smile ), I can not give the link to the selenium test here. However, if anyone is interested just let me know and I'll give you the link. Feedback and critique is then very much appreciated smile

Thanks.

Average of ratings: Useful (1)
In reply to Christian Lebe

Re: Moodle Selenium Test

by Martín Langhoff -
EHLO Christian!

I am interested in seeing your work on Selenium. It's probably a good ideat to get in touch with Rod Norfor @ open university . IIRC, he is (was?) working on something similar, with a modified PHPUnit, and he had a reasonably good set of tests for datalib functions.
Average of ratings: -
In reply to Martín Langhoff

Re: Moodle Selenium Test

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Selenium is more for GUI testing, rather than unit testing functions etc, so this is complementary to that.

These tests run in your browsers and are more like:

Click this button, check output, select this item, check output, etc

Average of ratings: Useful (1)
In reply to Martín Langhoff

Re: Moodle Selenium Test

by Nick Freear -
Dear Christian, Martins

At the Open University we have a Unit test framework built on Lastcraft's Simple Test (Martin L's post). This is similar to JUnit for Java, but written in PHP using version 4-compatible classes. The code to be tested can be class-based or procedural, and the documentation is good. There is support for grouping tests, mock objects and testing web pages at the browser level.

So far, the VLE coders have added 560 tests for new OU-Moodle code, which is very encouraging wink Unfortunately, we have not had the time/resource to retrospectively add many tests to the existing Moodle code base (datalib would be a prime target). This would make our code merges more straightforward, but is a difficult task sad

From a quick look at Selenium it is interesting - I agree with Martin D that Selenium complements, but doesn't replace unit tests.
Average of ratings: -
In reply to Nick Freear

Re: Moodle Selenium Test

by Martín Langhoff -
Hi Nick! Any chance of seeing how you are using it? Once the framework is in place, the "create test cases" task is more granular, and easy to add to organically.

Overall, with Selenium and/or Simple Test, I'm very keen on getting some test framework off the ground...
Average of ratings: -
In reply to Martín Langhoff

Re: Moodle Selenium Test

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Leaping in here ...

I suspect we won't have time to package this up for the community until May, once we have got our first ou moodle release finished.

It is probably not a lot of work, but there are some issues to sort out, like where to put the various files.

  • The testcases themselves we have put in folders called 'simpletest', with filenames like text_xxx.php, so you have a /lib/simpletest/testdatalib.php
  • Then there are the simpletest libraries, which we will have to move into a subfolder of lib.
  • Then there is the testing infrastructure, which basically crawls the directory tree (or a subset of it) looking for all the files called **/simpletest/test*.php, and runs the tests they contain (with various options) and displays the results prettily. At the moment we put this in a '_tools/runsimpletest' top level folder, but we may want to rethink that.
However, on the good news site, I have started writing some tests for datalib.
Average of ratings: -
In reply to Christian Lebe

Re: Moodle Selenium Test

by David Scotson -
I'd be interested in having a look. The email in my profile is valid.
Average of ratings: -
In reply to David Scotson

Re: Moodle Selenium Test

by Christian Lebe -
Hi David and everyone else,

Have you received my email about the Selenium test? Sent it around two weeks ago, no rush at all! just checking smile
Average of ratings: -
In reply to Christian Lebe

Re: Moodle Selenium Test

by David Scotson -

I had got the original email, just not had time to check it out till now, sorry.

After only a brief poke around I do have some preliminary questions:

  • could the tests clean up after themselves automatically, by deleting any activities created during the test (and either testing that they've gone or using standard setup and teardown routines to ensure a consistent environment for each test)?
  • to what extent do you think the cause of difficulties in testing is split between inherent problems with GUI testing / Selenium drawbacks / fixable Moodle HTML issues?
  • what other functional testing tools are in the running for selection?
  • what is the security issue with Selenium? Is it just to prevent people running scripts against test.moodle.com? Would it not be possible to run these tests against local installs of Moodle (if Selenium were set up on those servers too)?
  • has any thought been given to running these tests directly from the MoodleDocs wiki? Sounds crazy I know, but there's a precedent in Fit and Fitnesse.
  • have you been using the Selenium IDE or Remote Control?

I think it's looking very good, a comprehensive set of such tests would be an excellent resource for many reasons.

Average of ratings: -