25 - April - 2016

Codeception modules for Drupal acceptance testing

Post by Paul B Paul B

Over the last couple of years, we have been using Codeception at Ixis for running automated acceptance tests during development work. Over this time we attempted to distil some of the ideas and abstract custom code into Codeception modules, which are all available on GitHub.

Adopting Codeception for automated acceptance testing has been quite the learning process: getting used to the Codeception system, best practices and underlying code; facing challenges when writing tests for unusual or difficult situations; moving from the cURL-based PhpBrowser to the more advanced and complex WebDriver; problems with underlying components such as Selenium and PhantomJS; lack of integration with Drupal... phew! And that's not before we've considered processes, methods and best practices for running automating testing itself!

The latter, however, is a story for perhaps another time. This post summarises the Codeception modules we've worked on, why they might be useful and some lessons learned.

Drupal Content Type Registry

Drupal Content Type Registry is a module to provide a set of classes that encapsulate Drupal content types. This makes it much easier to quickly test standard Drupal functionality relating to content types, taking into account how they exist on your site. It enables testing of things such as the content types admin page, the 'manage fields' page for each content type, and provides createNode() and deleteNode() methods that can be used to quickly create test nodes where you can provide the test data using specific values, random values, or a range of values where one is picked at random.

Once enabled, a contentTypes.yml file should be created which defines:

  • GlobalFields - fields that will be used across all of the content types on the site. This is useful for things like title and body fields, to save you having to redefine the exact same field on every content type.
  • ContentTypes - content types on the site and the fields they have (global or otherwise).

Here's the example from the README:


GlobalFields:
    body:
        machineName:    body
        label:          Body
        type:           Long text and summary
        selector:       "#edit-body-und-0-value"
        widget:         Text area with a summary
        required:       true
    title:
        machineName:    title
        label:          Title
        type:           Node module element
        selector:       "#edit-title"
ContentTypes:
    news:
        humanName:    News
        machineName:  news
        fields:
            globals:
                - title
                - body
            field_image:
                machineName:    field_image
                label:          Image
                type:           Image
                selector:       "#edit-field-image"
                widget:         Media file selector
                required:       true
                testData:       "image1.png"
            field_icon:
                machineName:    field_icon
                label:          Icon
                type:           Text
                selector:       "#edit-field-icon"
                widget:         Text field
                skipRoles:
                    - editor
                    - publisher
                testData:
                    - smiley
                    - grumpy
                    - happy
                    - wacky
                preSteps:
                    - ["click", ["#button"]]
                    - ["fillField", ["#the-field", "the-value"]]
                postSteps:
                    - ["waitForJs", ["return jQuery.active == 0;"]]
        submit: "#edit-submit-me-please"

The definition of fields comprises the machine name, label, field type, widget and CSS ID or selector used when filling in node edit forms. Similarly, the definition of content types comprises the machine name, human-readable name and a list of field definitions. The module also provides a set of classes representing most field widgets provided by Drupal 7 core.

There are other features too, such as: preSteps and postSteps, which allow optional steps to run before and after filling the field; testData, which allows specific test data (literals or random text) to be used for each field; skippedRoles, which skips certain fields for specific user roles when creating nodes; and "Extras", which simulate the user clicking things on the node edit form that are not actually fields, like set the sticky status or the publication status of a node. For more detailed information on the module's configuration, see the README file.

This module (in combination with Drupal User Registry) also provides enough to create close to generic tests that will check your Drupal 7 site for all expected content types and fields. See this example Gist.

Credit must go to Chris Cohen for his original work on the Content Type Registry.

Drupal Drush

Drupal Drush allows the running of Drush commands in acceptance tests. It also allows the use of the following statements in tests:


// Execute "drush cc all"
$I->getDrush("cc", array("all"))->mustRun();

The getDrush() method returns and instance of Symfony\Component\Process\Process so you can read stdout, get the exit code, etc. The ->mustRun() method is useful as a Symfony\Component\Process\Exception\ProcessFailedException exception will be thrown if the command fails, meaning your test will automatically fail.

Drupal Mail System

Drupal Mail System allows the testing of the Drupal mail system.


// Test to see expected number of emails sent.
$I->seeNumberOfEmailsSent(1);

// Clear emails from queue.
$I->clearSentEmails();

// Check email fields contains text
$I->seeSentEmail(array(
    "body" => "body contains this text",
    "subject" => "subject contains this text",
));

Relies on TestingMailSystem class which stores the emails in a Drupal system variable.

Drupal Pages

Drupal Pages is a dependency of some other modules listed here and contains several PageObjects for "generic" Drupal 7 pages, based on "vanilla Drupal 7' and the default Bartik front-end or Seven administration themes. These PageObject classes can be extended to override any static properties as required for the site being tested.

Drupal User Registry

Drupal User Registry is a Codeception module for managing test users when running acceptance tests with Codeception. Once configured, it can automatically create and delete test Drupal users at the beginning and end of a test suite run. These users can then be used during acceptance tests to login and test elements of the project specific to that role.

The module is configured in the suite configuration:


class_name: AcceptanceTester
modules:
    enabled:
        - PhpBrowser
        - DrupalUserRegistry
    config:
        PhpBrowser:
            url: 'http://localhost/myapp/'
        DrupalUserRegistry:
            defaultPass: "foobar"
            users:
                administrator:
                    name: administrator
                    email: admin@example.com
                    pass: "foo%^&&"
                    roles: [ administrator, editor ]
                    root: true
                editor:
                    name: editor
                    email: editor@example.com
                    roles: [ editor, sub-editor ]
                "sub editor":
                    name: "sub editor"
                    email: "sub.editor@example.com"
                    roles: [ sub-editor ]
                authenticated:
                    name: authenticated
                    email: authenticated@example.com
                    roles: [ "authenticated user" ]
            create: true
            delete: true
            drush-alias: '@mysite.local'
  • defaultPass - use this password for all created user accounts, unless they have one individually specified.
  • users - a list of test user accounts to create, complete with username. email, password and a list of roles.
  • create and delete - whether to create and delete users at the start and end of a run.
  • drush-alias - the Drush alias to use when managing users via DrushTestUserManager.

Once configured, the module also allows the use of the following statements in tests:


// Returns a DrupalTestUser object representing the test user available for
// this role.
$user = $I->getUserByRole($roleName);

// Returns a DrupalTestUser object representing the test user available for
// exactly these roles.
$user = $I->getUserByRole([$roleName1, $roleName2]);

// Returns a DrupalTestUser object representing the user, or false if no users
// were found. Note this will only return a user defined and managed by this
// module, it will not return information about arbitrary accounts on the site
// being tested.
$user = $I->getUser($userName);

// Returns an indexed array of configured roles, for example:
//   array(
//     0 => 'administrator',
//     1 => 'editor',
//     2 => ...
//   );
$roles = $I->getRoles();

// Returns a DrupalTestUser object representing the "root" user (account with
// uid 1), if credentials are configured:
$rootUser = $I->getRootUser();

This module is used in any acceptance test suite we create in order to test specific elements where being logged in as a user with specific roles is necessary. However, there are some limitations and improvements that could be made.

The user registry can be used in combination with codeception Step Objects and Drupal Pages to provide login and logout functions to enable testing as an authenticated user. For example:


    $ php codecept.phar generate:stepobject acceptance AuthenticatedSteps

    class AuthenticatedSteps extends \AcceptanceTester
    {
        /**
         * Log in.
         *
         * @param DrupalTestUser $person
         *   The Drupal Person to log in.
         */
        public function login(DrupalTestUser $person)
        {
            $I = $this;

            $I->amOnPage(UserAccountPage::route('login'));
            $I->expectTo('not be redirected due to already being logged in');
            $I->seeCurrentUrlEquals('/' . UserAccountPage::route('login'));

            $I->expectTo('see various elements of the login page');
            $I->seeElement(UserAccountPage::$loginFormUsernameSelector);
            $I->seeElement(UserAccountPage::$loginFormPasswordSelector);
            $I->seeElement(UserAccountPage::$loginFormSubmitSelector);

            $I->amGoingTo('fill in the login form');
            $I->fillField(UserAccountPage::$loginFormUsernameSelector, $person->name);
            $I->fillField(UserAccountPage::$loginFormPasswordSelector, $person->pass);
            $I->click(UserAccountPage::$loginFormSubmitSelector);

            $I->expectTo('log in successfully');
            $I->dontSee(UserAccountPage::$loginFormCredentialsErrorMessage, Page::$drupalErrorMessageSelector);
        }

        /**
         * Log out.
         */
        public function logout()
        {
            $I = $this;

            $I->amGoingTo('log out');
            $I->amOnPage(UserAccountPage::route('logout'));
            $I->expectTo('be redirected to the front page');
            $I->seeCurrentUrlEquals('/');
        }
    }

Then in a test:


    /**
     * Test pages.
     *
     * @guy AcceptanceTester\AuthenticatedSteps
     */
    class ThePagesCest
    {
        /**
         * Test the page as an authenticated user.
         *
         * @param AuthenticatedSteps $I
         */
        public function testThePage(AuthenticatedSteps $I)
        {
            $I->login($I->getRootUser());
            // ...
            $I->logout();
        }
    }

There's potential for integrating the login(), logout() and other related functionality into the Drupal User Registry module itself, but still allow the test suite author to override or specify these procedures themselves.

Drupal Variable

Drupal Variable allows us to test Drupal system variables, for example:


// Assert that the target site has variable "clean_url" set to 1
$I->seeVariable("clean_url", 1);

// Set a variable.
$I->haveVariable("clean_url", 0);

// Delete a variable.
$I->dontHaveVariable("clean_url");

// Retrieve a variable value.
$value = $I->getVariable("clean_url");

This module was the key to working out different "connections" to Drupal itself. Previously, we were restricted by the assumption that everything needed to be done via the browser in acceptance testing (see the lessons learned around creating test content via the test browser, in Drupal Content Type Registry above).

Drupal Variable can connect to a Drupal site using one of three methods:

  • Bootstrapped - if the site is accessible on a locally mounted file system, fully bootstrapping the site is possible. In this case we use the Codeception\Module\Drupal\Variable\VariableStorage\Bootstrapped class and ensure we set the drupal_root configuration setting.
  • Direct connection - if the site is remote but we have access via an open MySQL connection, we use the Codeception\Module\Drupal\Variable\VariableStorage\DirectConnection class and set a dsn, user and password.
  • Drush - if the site is remote but access via a Drush alias is available, we use the Codeception\Module\Drupal\Variable\VariableStorage\Drush class and ensure we set the drush_alias setting in the module's configuration.

In review

Problems with creating test content in the browser

Creating lots of test content in the browser with Drupal Content Type Registry has proved problematic in places and in hindsight not the best way to achieve our goal. Many of our development projects and supported sites have additional elements that make things that little more difficult too, such as alternative publishing workflows with Workbench or WYSIWYG editing and media management with CKEditor and the Media suite of modules.

Whilst these are problems that must be tackled when explicitly testing the UI and user flow to verify a particular feature, it's not necessary to effectively repeatedly run this test in order to set up test data and content. Not only is it in places complex, but running a full suite (in multiple environments, i.e. in different browsers) is slow and a lot of that time is (obvious facepalm incoming) setting up test data.

Once we had a more 'useful' connection to Drupal with Drush, database or a fully bootstrapped connection as implemented in the Drupal Variable module, it became clear that there could be a better (or at least faster) way of setting up test content. We're not 100% what this will involve yet, perhaps custom code, or integration with modules such as Migrate or Node export.

Refactoring and relating the modules

Most of these module sprang from solving different problems whilst testing different projects. Whilst we have one or two suites using all if not most of them, the modules function together but may benefit from refactoring any shared code. For example, Drupal User Registry will currently check for, create and delete test users using Drush and aliases and the original intention was to have other ways of doing so via other connections. As mentioned above, Drupal Variable implements a different method but already has three types of connection, including Drush, which Drupal User Registry could utilise.

Behat?

Fairly recently Codeception introduced Gherkin and natural language features which is up there at the top of a list of advantages that Behat may have over Codeception - second only to it's much more extensive integration with Drupal, of course! Behat is being considered as an alternative or supplemental method for testing our Drupal sites.

Paul B

Paul B

Development Manager

Manager of all things development related at Ixis. Looking after the clients from discovery time to delivery of complete web builds.

Comments

Thank you for sharing this. I hope to start using Codeception for Drupal in a project soon.

JvE

Great information here, Paul!

I've put CodeCeption to the side for the last year or so, mainly because I haven't had the drive to put together all the additional code required to reap the full potential of the framework for testing a Drupal application - but I think I'll give it a go and try out some of the modules you've linked!

Adam Thomason

Add new comment

Share this article

Our thoughts

Let's work together

Get in touch and find out how we can empower your organisation.
Back to top