Hi Folks,
As many of you may be aware, the Integration team runs Behat daily across a wide range of databases, PHP versions, and browsers - for each supported version of Moodle.
We currently run against the following browsers:
- Goutte (for non-javascript tests);
- Firefox (via Selenium);
- Chrome (via chromedriver); and
- PhantomJS
Originally we made wide use of PhantomJS because chromedriver was less reliable, Safari had no reliable way to automate and we wanted a reliable way of testing WebKit browsers with Moodle. It was also faster than other drivers.
Since then much has changed:
- PhantomJS has ceased development
- ChromeHeadless is on it's way; and
- Safari 10 adds support for WebDriver.
In addition to these announcements, we've also found that PhantomJS is less and less reliable, and other browsers are becoming faster.
With all of these changes the Integration team is looking to shake up the browser testing matrix and we will be taking steps to stop testing with PhantomJS.
We are also looking to make ChromeHeadless available (MDL-58948), and to begin running tests on Safari 10 (MDL-56134). This will give us better coverage of both the WebKit, and the Blink engines.
For the short term we have already stopped running Behat tests against PhantomJS for all supported versions of Moodle.
In the medium-term we plan to remove support for running tests against PhantomJS from Moodle 3.4. This will be performed in MDL-59102.
Longer term we are planning to spend time ensuring that both Safari 10, and Chrome Headless can be easily tested against in time for the 3.4 release.
If you have any concerns about the removal of PhantomJS support in Moodle 3.4 we would be very keen to hear your concerns and comments - either in this discussion, or on MDL-59102.
Happy Moodling,
Andrew