Parallel behat runs on seperate Selenium nodes

Re: Parallel behat runs on seperate Selenium nodes

by Mark Johnson -
Number of replies: 4
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Thanks Rajesh, that's immensley valuable information.  Part of the problem I'm currently facing is that about 50% of the time, one of the threads will crash, leaving a run incomplete.

I currently do 4 parallel runs on a 4-core server, so I'll reduce that to 3 as that may be causing the instability.  I already follow that with a single-thread rerun, and am working on rerunning any failures from that in isolation.

I'll take a look at doing the separate selenium nodes on Xvnc.  I suspect the method for that will be to add the selenium command to the ~/.vnc/xstartup file, and changing the port number before each vncserver command.

Average of ratings: -
In reply to Mark Johnson

Re: Parallel behat runs on seperate Selenium nodes

by Mark Johnson -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

OK, I'm making progress.  We run a Selenium hub process and have script (selenium_node.sh) for creating a new node, with the port number corresponding to the current $DISPLAY number.

To achieve parallel runs in separate Xvnc sessions, I've added a call to the selenium_node.sh in the ~/.vnc/xstartup file (this is the file run by vncserver on startup, but it's location differs on other distros).  I run an instance of vncserver (which launches Xvnc) for each run, and the config.php then has the $CFG->behat_parallel_run array described by Rajesh added.

This works with Firefox, I can VNC in to the 2 seperate displays and watch the two windows run seperate tests.  However, I can't get this to work with Chrome (we find Chrome significantly faster for running Behat tests).  This is due to the fact that the wd_host setting in the per-run behat.yml is only replaced in the default section, not in the chrome section.  Is it possible to have it set for Chrome/other profiles as well?

Average of ratings: -
In reply to Mark Johnson

Re: Parallel behat runs on seperate Selenium nodes

by Rajesh Taneja -

Thanks good to know that you managed to run FF.

If you remove wd_host from you chrome profile then it will work for chrome as well.

$CFG->behat_config = array(
    'chrome' => array(
        'extensions' => array(
            'Behat\MinkExtension\Extension' => array(
                'selenium2' => array(
                    'browser' => 'chrome',
                )
            )
        )
    )
);
What happens is if you define wd_host in profile then it take preference over the default wd_host  or one defined in $CFG->behat_parallel_run.


Hope this helps.

Cheers

rajesh

Average of ratings: Useful (1)
In reply to Rajesh Taneja

Re: Parallel behat runs on seperate Selenium nodes

by Mark Johnson -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Thanks Rajesh, that does appear to work.

Unfortunately, I've implemented my reruns in a bit of a silly way which makes this a bit problematic.  Hopefully refactoring in line with how you've done it will fix this.

In general, I've found that reducing my parallel runs to 3 and implementing reruns has created a much more reliable test process. Thanks very much for your help.
Average of ratings: -
In reply to Rajesh Taneja

Re: Parallel behat runs on seperate Selenium nodes

by Mark Johnson -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Hi Rajesh,

I got it working, thanks for your help.  Looking at your scripts was particularly useful.

I am now experiencing another issue with Selenium appearing to die in between re-runs.  However, I'll start a seperate thread to discuss that.

Average of ratings: -