Hi,
A quote from https://docs.moodle.org/dev/Acceptance_testing#Fixtures :
[...] Moodle provides a way to quickly set up the contextual data [...] this can be done using one of the site templates (TODO) or creating entities in the background section [...].
Do you know what those "site templates" refer to? Is that some functionality planned?
Do we have someone here with experience of doing Moodle behat tests for the sites where you had to set up a lot of initial data?
Tomek
Hi Tomasz,
This was one of the ideas that came from the Perth 2012 hackfest (https://docs.moodle.org/dev/Perth_Hackfest_October_2012) but wasn't finally implemented and I am not aware of any plans to do it soon.
If you need to set a lot of initial data for many different tests I would suggest you to create a step definition and set everything up in there, you could even use PHP data generators although better to avoid custom APi calls that could lead to differences between the site you are testing with behat and a real site.
Thanks for an explanation and for updating the wiki page.
I was also thinking about creating a new behat cli command that one could use to dump/serialize behat (b_) tables - to overwrite those created during php admin/tool/behat/cli/init.php run.
So my workflow would be:
1. Initialize tests.
2. Set up test data on the test site by whatever means (with scripts, generators, moosh or even manually).
3. Dump DB and data that will be used for restore after each test run.
4. Run tests.
Do you think that makes sense?
I think this is fundamentally opposed to the philosophy of Behat.
My understanding is that each Behat scenario should test one thing, and it should do that following the Four-phase test pattern.
When it comes to "tests help document how your feature works", the documentation is better if the set-up is minimal. Then it is clear what needs to be set up, for a particular bit of functionality to work. Anything else is a distraction.
So, the way Moodle works, with every test starting from the site as it was after a clean Moodle install is good.
If you have a big, complex, standard inital condition, wiht lots of demo courses with lots of activities, you would not know which bits of that were actually relevant to each feature being tests.
If you have a big, complex, standard inital condition, wiht lots of demo courses with lots of activities, you would not know which bits of that were actually relevant to each feature being tests.
Not at all - if I do the initial setup, I know exactly what it is.
We are talking here about the initial setup of a platform, I will call it "My Moodle". So in My Moodle, my users have courses in 10 top-level categories. Always - that's how their Moodle is set up from the very beginning and their categories are there, set in stone. My users know exactly what goes into each category - they are basically based on the core business model the organization has.
If those categories are not set up initially, then in each test I must add something like:
...And categoryX exists..."
It may make sense when you're writing tests for clean Moodle but for "my users" of "their Moodle" it is just a distraction.
"Any additional givens are distracting, which makes it hard for someone looking at the story for the first time – whether from the technical or business side – to understand what they need to know." - https://dannorth.net/whats-in-a-story .
The philosophy of BDD (behat included) is "writing software to achieve business outcomes" - https://dannorth.net/whats-in-a-story/ is a good read. Things like "Four-phase test pattern" mean nothing to the end users.
I honestly see a very strong case to support this kind of initial setup - so I can talk with end users about the tests for "their Moodle" and not tests like "take vanilla Moodle, then add all the things we set up for you". The communication & mutual understanding is the high priority for me.
I've just realised that all my behat testing is using that server rather than the local one, and I'm now saturating that server, locking out not only myself but anyone else attempting to trial CodeRunner. Is there any convenient way of setting the administrator's plugin settings for CodeRunner during behat testing, please?
Richard
Specifically for CoderRunner, I made this mechanism:
https://github.com/trampgeek/moodle-qtype_coderunner/blob/master/tests/coderunnertestcase.php#L51
Looking at that code, it looks to me like it should default to using localhost as the sandbox.
I am not sure this is a very good solution. It was just something that I could make work. I would be interested to hear if anyone has a better way to handle this sort of thing.
That mechanism should work for Behat too. If not, I screwed it up.
