Behat for dummies?

Behat for dummies?

Diane Soini -
回帖数：14

I have attempted to get behat testing up and running. I got as far as being able to run the init script. After that, nothing else works.

It appears to me that the init script simply runs the install script as the output looks similar to what you see when you run the installation through the web.

I want to run the tests to make sure things are working. None of the --rerun or any of the other flags work. I get this error:

  [Behat\Testwork\ServiceContainer\Exception\ConfigurationLoadingException] 
  The requested config file does not exist


What is the configuration file I need to create? I just want to run all the .features tests that come with moodle.

平均分： -
回复Diane Soini

Re: Behat for dummies?

Diane Soini -

Seriously, I cannot figure this out with any of the available online help.

I have a system on centos 7 running on a cloud server. I have moodle already installed and I can reach it with a browser via https://my.ip.address/path. The behat test system requires there to be a different wwwroot from the moodle site. I'm not sure how that works.

I've tried duplicating the entire moodle directory into another directory so that I can have a behat wwwroot that really is different and that is actually reachable in a browser. Not sure if that's what you are supposed to do.

The init script seems to work. It creates database tables. It spits out output to the screen that resembles the install process. It sticks some things into the behat dataroot directory.

I edited the behat.yml and added all the locations of all the feature tests that come with moodle:

default:
  suites:
    default:
      paths:
        - %paths.base%/user/tests
        - %paths.base%/theme/boost/tests/behat
        - %paths.base%/repository/recent/tests/behat

(etc)

Every time I try to run the test I get the following error:

php admin/tool/behat/cli/run.php
Running single behat site:

                                                                            
  [Behat\Testwork\ServiceContainer\Exception\ConfigurationLoadingException] 
  The requested config file does not exist                                  
                                                                            

To re-run failed processes, you can use following command:
php admin/tool/behat/cli/run.php --rerun

I've tried:

php admin/tool/behat/cli/run.php --config=/path/to/behat.yml. Same error.

I've tried putting behat.yml into the datadir. I've tried everything I can think of. I even tried just running behat directly as a command-line tool (i.e. vendor/behat/behat/path/to/behat) but all that does is spit out the text of all the lines in the feature tests and output sometimes that looks like this:

2 scenarios (2 undefined)
9 steps (9 undefined)
0m0.02s (12.56Mb)

All I really want to do is run all the test so I can see if merging in the latest stable moodle release branch was successful and identify any place I might have messed up resolving git conflicts.

Is there a step-by-step for dummies on how to do this?


平均分： -
回复Diane Soini

Re: Behat for dummies?

Tim Hunt -
Core developers的头像 Documentation writers的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像

Getting Behat working the first time is a pain, however, it should not be that hard.

Have you worked through the 'Short version' here: https://docs.moodle.org/dev/Running_acceptance_test#Short_version

The other option is some of the docker containers linked at the bottom of that page, if you know about Docker.

The good news is, once you get it working, it tends to keep working.

平均分：Useful (1)
回复Diane Soini

Re: Behat for dummies?

Marcus Green -
Core developers的头像 Particularly helpful Moodlers的头像 Plugin developers的头像 Testers的头像

I second Tims comments. This link may help

https://gist.github.com/danpoltawski/8036489


平均分： -
回复Marcus Green

Re: Behat for dummies?

Diane Soini -
Thank you. That at least was a new one. But I have no luck at all.

$CFG->wwwroot   = 'https://my.ip.address/path';

$CFG->behat_wwwroot = 'https://127.0.0.1/path';
$CFG->behat_prefix = 'bht_';
$CFG->behat_dataroot = '/local/behat';

nohup java -jar selenium-server-standalone-3.14.0.jar &

php admin/tool/behat/cli/init.php
(Installs a bunch of stuff, then...)
-->System
...
++ Success ++
-->antivirus_clamav
++ Success ++
-->availability_completion
++ Success ++
-->availability_date
++ Success ++
-->availability_grade
....

vendor/bin/behat --config /local/behat/behat.yml
No scenarios
No steps
0m0.01s (12.25Mb)

If I run the init again, it says:

php admin/tool/behat/cli/init.php
You are already using composer version 1.7.3 (stable channel).
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Nothing to install or update
Generating autoload files
Unknown error 1 This is not a behat test site!


平均分： -
回复Diane Soini

Re: Behat for dummies?

AL Rachels -
Core developers的头像 Particularly helpful Moodlers的头像 Plugin developers的头像 Testers的头像

I suspect that your dataroot is wrong.

$CFG->behat_dataroot = '/local/behat';


It should be almost identical to your regular Moodle dataroot. e.g. for my Centos Linux I have this.

$CFG->dataroot  = '/var/moodledata/moodledataxx';
$CFG->behat_dataroot = '/var/moodledata/bht_moodledataxx';

平均分： -
回复AL Rachels

Re: Behat for dummies?

Diane Soini -

My moodle data is at /local/courses

Inside of that are the standard moodle data directories:

cache  filedir  lock  models  muc  simplerestore  temp  trashdir

My behat data is at /local/behat

Inside of that are the following directories:

behatrun  run_environment.json

Does that look wrong?

平均分： -
回复AL Rachels

Re: Behat for dummies?

Diane Soini -

One thing it says is: $CFG->behat_wwwroot needs to point to your Moodle site yet be different from the 'normal' wwwroot (e.g. if you used localhost for wwwroot use 127.0.0.1 for the behat_wwwroot). Whatever you choose, make sure it works.

I have:

$CFG->wwwroot   = 'https://server.ip.address.available.to.outside.world/gs_35';

$CFG->behat_wwwroot = 'http://127.0.0.1/gs_35';

The thing is, it doesn't work.

wget 127.0.0.1/gs_35

--2018-11-02 17:33:50--  http://127.0.0.1/gs_35
Connecting to 127.0.0.1:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://127.0.0.1/gs_35/ [following]
--2018-11-02 17:33:50--  http://127.0.0.1/gs_35/
Reusing existing connection to 127.0.0.1:80.
HTTP request sent, awaiting response... 500 Internal Server Error
2018-11-02 17:33:50 ERROR 500: Internal Server Error.


wget 127.0.0.1 works though.

平均分： -
回复Diane Soini

Re: Behat for dummies?

Diane Soini -

Maybe can someone show me where there are instructions for editing the behat.yml file? There are none in any of the moodle documentation. It's not clear to me if you are supposed to put a list of all the paths to the .feature files or not, if you are supposed to change selenium2: null to something, if moodledirroot: is the same as $CFG->wwwroot or is the same as  $CFG->behat_wwwroot, if steps_definitions: {  } is supposed to remain empty, etc. Nowhere does it say where this file is supposed to live.

Also, no where is it explained whether moodle's built in .feature files actually function. Do you have to do something to make them work?


平均分： -
回复Diane Soini

Re: Behat for dummies?

Diane Soini -

I think I found my problem.

We have some custom blocks that were written long ago with some of the sloppier php conventions of that era still present. (e.g. using variables that have not yet been declared and things like that.) This only produces PHP Notices, not fatal errors, but the whole behat init step treats them like fatal errors and so the init step never finished.

I deleted those custom blocks and ran init and it completed fully and created the behat.yml file for me.


Next problem:

I attempted the run and found a bug in blocks/tests/behat/behat_blocks.php. This line was duplicated and caused a fatal error:
use Behat\Mink\Exception\ElementNotFoundException as ElementNotFoundException;

I removed the duplicate line and ran the run test.


Next problem:

Behat requirement not satisfied: http://127.0.0.1/gs_35 is not available, ensure you specified correct url and that the server is set up and started.


I'll see if I can figure that part out now.

平均分： -
回复Diane Soini

Re: Behat for dummies?

Diane Soini -

So I'm pretty stuck now on the path I need to use for $CFG->behat_wwwroot

$CFG->behat_wwwroot = http://127.0.0.1/gs_35

php admin/tool/behat/cli/run.php --rerun

Running single behat site:
Behat requirement not satisfied: http://127.0.0.1/gs_35 is not available, ensure you specified correct url and that the server is set up and started.
 More info in http://docs.moodle.org/dev/Acceptance_testing#Running_tests


If I curl http://127.0.0.1/gs_35 I get a 301 redirect to http://127.0.0.1/gs_35/

http://127.0.0.1/gs_35/ redirects to https://127.0.0.1/gs_35/.

https://127.0.0.1/gs_35/ redirects to the value of $CFG->wwwroot.

平均分： -
回复Diane Soini

Re: Behat for dummies?

Diane Soini -

On this page: https://docs.moodle.org/dev/Running_acceptance_test#2._Set_up_Selenium

How do you do this part?

check it works, access 'localhost:4444/wd/hub/' in your browser and check you can create a new Chrome session.

I tried using wget and I got a page with some javascript, but I don't know if that qualifies as "works".

When I run the behat tests I get an error:
Selenium server is not running, you need to start it to run tests that involve Javascript. More info in http://docs.moodle.org/dev/Acceptance_testing#Running_tests

平均分： -
回复Diane Soini

Re: Behat for dummies?

Diane Soini -

Well, I give up.

I successfully installed the acceptance site:

Acceptance tests site installed
Acceptance tests environment enabled on http://my.ip.address/gs_35, to run the tests use:
vendor/bin/behat --config /local/behat/behatrun/behat/behat.yml


But I can't get past the following error:

Behat requirement not satisfied: http://my.ip.address/gs_35 is not available, ensure you specified correct url and that the server is set up and started.
 More info in http://docs.moodle.org/dev/Acceptance_testing#Running_tests


If I go to that URL with a browser, I see a page with the following:

<!DOCTYPE html>
<title>WebDriver Hub</title>
<link rel="stylesheet" href="style.css">
<script src="client.js"></script>
<body>
<script>init();</script>
</body>

Same thing if I curl it from a terminal window logged in on the system, so I know it is accessible from my Eucalyptus instance.

But for whatever reason, behat or Selenium or something about this whole thing just doesn't work.


平均分： -
回复Diane Soini

Re: Behat for dummies?

Diane Soini -

So I finally got it running, at least in a crippled form. I am using the php -S localhost:8000 means of running it. That requires excluding all the javascript tests.

$CFG->behat_wwwroot = 'http://localhost:8000';

Then in one terminal window, I ran:

php -S localhost:8000


And in the other window I ran:

php admin/tool/behat/cli/util.php --enable

vendor/bin/behat --config /local/behat/behatrun/behat/behat.yml --tags="~@javascript"

It appears all the tests are failing. It appears they can't get past any kind of PHP notice errors. Well, that's at least something.

平均分： -