General developer forum

Indeed! Thanks for the URL.

Are these tests intended ever to be run by regular admins, or only ever by developers?

If they're only intended for developers, I don't think it makes sense to have them in HEAD, because it's just extra stuff sitting around taking up space and bandwidth and waiting to be exploited.

Since I started thinking about this problem just a few hours ago, I had already considered and rejected the option of having a test infrastructure that crawls a directory structure. But I guess I don't have any strong objections to it.

I do think we really need some way of grouping tests and perhaps indicating some level of granularity. Some tests are high-level and some are super-anal, and the rest are in-between. I don't need to run 3 hours of super-anal tests every day, but it would be helpful to run 5 minutes of high-level tests regularly and then run the super-anal set just before I check code into CVS.

If all of your tests are at or near a particular point on the high-level->super-anal continuum, it isn't an issue now. But different devs will write tests at different levels of granularity, and the pool of tests will diverge from the happy medium you've probably struck now, and the "long time" your tests take to run will just get longer and longer and longer and...

These are my guesses, anyway. "I'm not a QA guy, but I play one in the forums sometimes..." In any case, I did talk with a friend about software testing in his company, and his major complaints about their situation have inspired this concern about levels.

It seems like it wouldn't be too hard to add some indication of granularity level to the tests, once we define the number of levels and have some indication of what they mean.

Hi,

Messaging me is the best way. More options in my profile.

Why *simple*test? I'd rather the names were simplified down to 'test', something like:

lib/testlib.php

lib/test/testdatalib.php

mod/forum/test/testlib.php

I'm probably slightly in favour of keeping the (redundant) 'test' prefix on the filenames to avoid possible confusion with the originals and to allow non-test files to also be put in those directories, but if these aren't really advantages then let's drop that too.

OK, having written the post I got curious and tried it. It does fit in directly within the existing test framework. Here's an example:

require_once('simpletest/web_tester.php');

class test_learn_website extends WebTestCase {
    
    const USERNAME='??',PASSWORD='??',SITE='d820-06e',
        BASE='http://learn.open.ac.uk/';

    function setUp() {
    }
    
    function test_sams() {
        $this->get(self::BASE.self::SITE);
        $this->assertTitle('Sign IN');
        $this->setField('username',self::USERNAME);
        $this->setField('password',self::PASSWORD);
        $this->clickSubmit('Sign in');
        $this->assertTitle('Course: The Challenge of the Social Sciences');
    }
    
}

Username and password changed to protect the innocent [It was a dummy account, not mine. Even so.]

This tries to log into a Moodle course (course/view.php) via an Apache mod_redirect, which results in getting redirected to our single sign-on page and then redirected again about 387 times [unless they've changed it - I had to manually implement something that went through the process once and it was hugely tedious]. Then I made it fill in the username and password and sign in, which ends up creating some new cookies and (eventually) redirecting back to the Moodle site, automatically logging into that via our auth method, and viewing the course page.

Not very difficult I think! And neat that it can run in the same automated framework as our other tests. So anyway it's pretty cool, although obviously it doesn't handle Javascript, or browser-specific testing.

--sam