The behat.yml in 2.8 lists under features the root features folder in each plugin and in runtime all features are executed including those in sub folders. In 2.9 it lists particular features but only from the root features folder and only those features are executed. Adding the root features folder to the behat.yml fixes that but only until the next initialization. Bug? Known? Or am I missing something?
In reply to Itamar Tzadok
Re: Behat cli doesn't execute features from sub-folders in 2.9?
by Tim Hunt -
What do you mean 'root features folder'?
The place you are supposed to put .feature files is
path/to/plugin/tests/behat/something.feature
and that can work even with quite a lot of .feature files: https://github.com/moodle/moodle/tree/master/mod/quiz/tests/behat
What file layout are you trying to use?
In reply to Tim Hunt
Re: Behat cli doesn't execute features from sub-folders in 2.9?
by Itamar Tzadok -
The place we are supposed to put .feature files is
path/to/plugin/tests/behat
and while this can work with many feature files, in some cases it may be more manageable to add a logical division within that place, e.g.
path/to/plugin/tests/behat/cat1
path/to/plugin/tests/behat/cat2
path/to/plugin/tests/behat/cat1/cat1.1
etc.
This is possible in 2.8 b/c there the yml lists the feature places (folders) rather than particular feature files. In 2.9 it lists particular files but it doesn't take files from sub-folders.
In reply to Itamar Tzadok
Re: Behat cli doesn't execute features from sub-folders in 2.9?
by Tim Hunt -
I don't know why they did that. I had not realised they had changed it.
Digging into git history, it looks like it was done as part of MDL-39752, which is a huge win for the future, so I think this is a price worth paying.
If you want a simple work-around, just make a naming patter like I did for the quiz tests, with several related .feature files with names that start the same, like attempt_{details}.feature or editing_{details}.feature. You can also group several related features into each file.
In reply to Tim Hunt
Re: Behat cli doesn't execute features from sub-folders in 2.9?
by Itamar Tzadok -
I've already tried both strategies. Both don't seem to be satisfactory as the tests keep piling up. I wonder if its not just simply overlooking the possibility of sub-folders. Afacit MDL-39752 shouldn't have any problem with taking feature files from sub-folders when compiling the list. I'll raise the point in the tracker.
In reply to Itamar Tzadok
Re: Behat cli doesn't execute features from sub-folders in 2.9?
by Rajesh Taneja -
Thanks Itamar and Tim for raising this.
This is related to MDL-39752, as we are now specifying each feature file as part of this fix, so they can be divided for parallel runs.
Moodle by default doesn't support feature files to be placed under sub-directory. If you want to include more feature files, you should use $CFG->behat_additionalfeatures.
It was working before, because we used to just set file paths and behat was recursively searching for feature files within those paths.
In reply to Rajesh Taneja
Re: Behat cli doesn't execute features from sub-folders in 2.9?
by Itamar Tzadok -
Thanks. the config setting indeed worked around this. But I'm not sure why it is not possible to register features from sub-folders? If parallel execution relies on full file paths in the yml, then, presumably, all that is required is to read on behat init all sub-folders if exist and add the file paths to the yml. No?
In reply to Itamar Tzadok
Re: Behat cli doesn't execute features from sub-folders in 2.9?
by Rajesh Taneja -
Sure, it is possible.
Only point is, we never officially supported/encouraged it. So should be actually support different paths other then what moodle recommends.
I have added this point for the follow-up issue in https://tracker.moodle.org/browse/MDL-49456. Please feel free to put your point forward so it can get in moodle code.