It seems that from v48 onwards, Firefox will no longer support Selenium's own FirefoxDriver and will use Mozilla's Marionette for interfacing with WebDriver instead. There are currently very few instructions around for getting Marionette working with Selenium (let alone Behat), has anyone had any success with this that they'd be able to share? Is Marionette on the radar for Moodle HQ's QA processes?
Hi Mark,
yes, we are well aware of incoming Marionette and FF 48.
Right now, and since some weeks ago, we are pushing hard to get current Selenium 2.53 and (latest*) FF 46.0.1 running ok under all OSs so we can declare it to be a working combination.
So, our current goals are:
- For Moodle < 3.1 => Selenium 2.47.1 and FF 41 (no way we can backport anything newer there easily).
- For Moodle >= 3.1 => Selenium 2.53 and FF 46.0.1
And whenever both FF 48 is available and Marionette is ready, we'll start the move to it, indeed. At what point that will happen, we have not all the information yet, but only will declare it to be a working combination when ready. Till then, the combinations above are the supported ones.
Finally, and not less important, this is the issue where Selenium 2.53 + FF 46.0.1 are being fixed and tested: MDL-54589 (it seems that we have, finally, fixed all the issues, running the last battery of tests right now).
Ciao
(latest*): Note I told FF 46.0.1 was latest FF version, because there is a bug in FF 47 causing it to break badly with Selenium 2.53. If incoming FF 47.0.1 fixes it... we'll consider it.
That's great Eloy, thanks for the info.
I've been having trouble running Behat tests which were working five 5 months ago (with a different version of Ubuntu), and I've just stumbled upon this thread. Could the footnote to the last posting above explain my problems, do you know?
I'm running Ubuntu 15.10/Firefox 47.0. When I try to run my tests I get the following output in the command window (even though selenium-server is running - see the attached log):
rjl83@rjl83-eb:/var/www/html/moodle$ vendor/bin/behat --config /var/behat/behat/behat.yml --tags @questiontype_coderunner Moodle 3.1+ (Build: 20160616) Php: 5.6.11.1.3.3, mysqli: 5.5.49-0ubuntu0.14.04.1, OS: Linux 3.16.0-70-generic x86_64 Server OS "Linux", Browser: "firefox" Started at 24-06-2016, 10:05 There was a problem with Selenium or your browser. Please ensure you are using the latest version of Selenium. Error: Selenium server is not running, you need to start it to run tests that involve Javascript. More info in http://docs.moodle.org/dev/Acceptance_testing#Running_tests
In the window in which I launched selenium-server, I get prodigious error messages - see the attached error log.
I've run simple Python-based tests that connect to selenium-server + Firefox and they work fine.
Any suggestions gratefully received.
Richard
Hi Richard,
as commented above FF 47.0 came with a bug causing Selenium to stop working, that's the reason we have stablished FF 46.0.1 like the latest working combination.
Coincidently, yesterday, Selenium released 2.53.1 that comes with a new selenium-firefox driver, ready to work with the, also new, FF 47.0.1
For the records we are testing that combination (Selenium 2.53.1 + FF 47.0.1) right now @ MDL-55094 . If it's probed to work and doesn't break any other browser... we'll add it as a working combination to the Docs, so everybody knows which ones are supported.
I really would recommend people to stay sticky to those combinations, because it's really hard to verify each one, and along the last year it has been a nightmare to keep things working, with every update of Selenium or Firefox causing new problems. That's pretty much the reason we decided to create that list of working combinations.
Ciao