We’ve been doing some work behind the scenes to update Behat in Moodle. Those changes should be applied to all stable versions of Moodle in the weekly releases this week and I’d like to let you know a little about what that might mean for you.
The details of this issue are in MDL-66979.
What is changing?
Behat is made of many moving parts. Some of those can be swapped out for different implementations as required.
We will be changing two parts of the Behat implementation that we use:
- we are migrating from
Behat\Mink\Driver\Selenium2Driverto a new MinkDriver (
- we are moving from
instaclick/php-webdriverto an updated implementation (
Why does it need to change?
The new MinkDriver includes a few minor changes over the existing driver, and uses a new php-webdriver, which is actively developed, regularly updated, and is much more widely used outside of Mink. These changes will also allow us to test with more recent versions of Firefox (we were previously stuck on Firefox 47.0.1), and to test with MS Edge. It also allows use of 'headless' browsers, and to use the browser drivers directly without using Selenium if preferred (
How will this change effect me?
We have been making small changes for the past few months to try and avoid any potential problems, or to at least surface them earlier. For the vast majority of developers there should be minimal-to-no change, and no breakage. There are a few areas which may affect you Behat configuraition, browser configuration, and custom step definitions.
Behat configuration may require changes in the following areas:
- it is strongly recommended that you should not have a copy of your
behat.ymlin the Moodle code directory. If a stale copy exists, then it can be used in some situations during configuration and you may see confusing and unusual error messages
- if you make use of the
$CFG->behat_configconfiguration to specify additional browsers, then you will need to update it to use
webdriverinstead. All other configuration should be identical
If you have been testing using Firefox, and have been using Firefox 47.0.1, or a docker image which provides it, then you will need to upgrade or use a newer image. If you are using docker, then the
selenium/standalone-firefox:latest image should be sufficient.
If you are using Chrome, there is a known bug with Chrome versions 80 - 89 whereby an element near the edge of the display is reported as being not-interacable. This has been fixed version 89. This issue also affected existing versions of Moodle and is unrelated to this change.
A new version of the moodle-plugin-ci tooling will be released in the coming days to ensure that Firefox testing uses the correct image depending on the version of Moodle installed.
We will be updating the Moodle-docker toolkit over the coming days too.
Most of the changes that we’ve been making so far have been fixes to the way in which we use existing features. For example, one of these changes was to standardise the way in which Behat interacts with the “autocomplete” element (MDL-69107), and another change was to standardise the way that Behat emulates keyboard interactions (MDL-70148). Some of the fixes relate to misuse of existing functionality, for example MDL-70167 dealt with testing the value of a boolean HTML attribute (boolean attributes should not have a specific value).
However, if you have written a custom step, and that custom step interacts directly with the MinkDriver, or the
php-webdriver implementation, then you will need to update your code. Generally speaking we strongly recommend against interacting directly with the drivers in a step definition. This should only affect a minority of people.
Documentation on the changes required are available at https://docs.moodle.org/dev/Behat_Driver_Update. If you encounter additional changes, please help others by documenting them.
You may also be affected if you are directly interacting with elements, using function such as
$element->click(). This is usually because the newer drivers are much faster. Moodle has a number of value-added features to ensure that when you interact with an element we wait for that element to have finished having its affect before we move to the next step. When you interact directly with elements and do not use our features, the browser does not know that Moodle is still busy. Our advice has never been to use the
->click() function directly, but to use the
$this->execute("behat_general::i_click_on", array(get_string('continue'), 'button’)); helper instead. The
The technical details
Behat has several layers, starting at Moodle and ultimately ending at the browser. One of those layers is the “MinkDriver”, which is responsible for translating the Behat commands into actions for your web browser. The Driver that we use further delegates the actual browser responsibility to a
For the past few years Moodle has been using the official Behat MinkSelenium2Driver, which itself uses
instaclick/php-webdriver. Sadly both of these projects have not been updated in any significant way in several years, and the
php-webdriver does not work with newer versions of PHP. Moodle has submitted a number of pull requests which have been ignored.
instaclick/php-webdriver library was originally written by Facebook, but forked by instaclick who have since abandoned it. The original Facebook library is now actively developed again, but has been renamed to
php-webdriver/webdriver. Moodle has been active in its development efforts over the past 18 months and are regular contributors to the project. Moodle has also been working on a new Mink Driver, written by Oleg Andrayev.
In addition to the abandoned nature of the drivers, the instaclick/php-webdriver was written at a time when browsers all implemented a standard called JsonWire. This standard has now been deprecated and is in the process of being replaced by a new standard, the W3C WebDriver.
The new drivers support both JsonWire, and W3C implementations, though actively prefer the W3C implementation.
Some other cool things
I’ve been working on a browser configuration helper, moodle-browser-config which can be included in Moodle’s
config.php and will generate a large range of common browser configurations for Behat.
That project is still under development, but is relatively stable and I hope to release a first official version in the coming weeks. It includes configurations out of the box for Chrome, Firefox, and Edge, including a headless variant, and variants both with and without selenium. Browserstack support is also included.
Please note: Safari support is included, but there are known bugs with the Safari Driver and this is not considered supported.
We may look at including this functionality into core at some point in the future, but that has not yet been discussed.
- Most things should stay the same
- If you're doing unusal things, then you may have to make some changs
- Check out moodle-browser-config for simplified cross-browser testing
- Upgrade your Firefoxes
Best wishes, and happy testing!