General developer forum

 
 
Picture of David Monllaó
Moodle automated functional tests
Group DevelopersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup Testers
Hi,
 
At HQ we are working on automated functional tests (acceptance tests) to periodically test all of Moodle's functionalities in all sort of OS and browsers combinations including some mobile devices, more of less what we have been doing manually with the QA tests. These tests are based on a behaviour driven development methodology and are written in a human-friendly language so not only developers can write them; this is an example of how one of these tests looks:
 
  Feature: Add activities to courses
    In order to provide tools for students learning
    As a teacher
    I need to add activites to a course
 
    Scenario: Add an activity to the course
      Given the following "courses" exists:
        | fullname | shortname | format |
        | Course 1 | C1 | topics |
      And I log in as "admin"
      And I follow "Course 1"
      When I turn editing mode on
      And I add a "Database" to section "3" and I fill the form with:
        | Name | Test name |
        | Introduction | Test database description |
        | Required entries | 9 |
        | Comments | Yes |
        | ID number | ASD123 |
      And I turn editing mode off
      Then I should not see "Adding a new"
      And I follow "Test name"
      And I follow "Edit settings"
      And the "Name" field should match "Test name" value
      And the "Required entries" field should match "9" value
      And the "Comments" field should match "Yes" value
      And the "ID number" field should match "ASD123" value
 
    Scenario: Add an activity without the required fields
      Given the following "courses" exists:
        | fullname | shortname | format |
        | Course 1 | C1 | topics |
      And I log in as "admin"
      And I follow "Course 1"
      When I turn editing mode on
      And I add a "Database" to section "3" and I fill the form with:
        | Name | Test name |
      Then I should see "Adding a new"
      And I should see "Required"

Here you can see a video of this tests running in an iPad using the SauceLabs infrastructure: https://saucelabs.com/tests/1c7e76eda791489fa8e144247b450d82
 
Moodle 2.5 will have a user interface to ease the tests creation, all the documentation can be found here http://docs.moodle.org/dev/Acceptance_testing. This framework is not limited to Moodle core codebase, add-ons can create their own steps (the PHP code that simulates user actions) and their own tests, using the base Moodle provides, which includes a test site to isolate the tests exeuctions from the production site, data generators to set up the tests contexts, and general steps to interact with form elements or to assert page contents.
 
This is currently available on the development branch (master) of Moodle and is based on Behat, the HQ servers are running the tests during the integration process to detect issues and regressions and contribute to Moodle robustness; during these next weeks more related issues will be integrated to refine the tests results, find more potential problems and allow the tests to run in more combinations of operative systems and browsers, including iPhone, iPad and Android. The next milestone is to automate as much QA tests as we can using this approach.
 
 
Average of ratings:Useful (7)