An Introduction to Unit Testing (for WordPress)

If you are a software developer, you might have come across the term “testable code”. But what is it? What makes code testable, and what not?

Testable Code

Every piece of software is testable—somehow. There are several things you might be able to test:

• the return value of a function;
• the output of a function;
• other side effects of executing a function;
• whether or not a program crashes;
• …

So, doesn’t that mean every piece of code is testable code?

It does not. That’s because almost always when someone refers to “testable code” they do it in the context of unit testing. So in order to explain how to write testable code, we first have to ensure a common understanding of unit testing. And this is what this post is all about.

Unit Testing

Unit testing—also known as component testing or module testing—is dynamic testing of individual units in isolation.

“Oh, thanks. Now that really helped. Not.”

OK, OK, got it. Well, then let’s tackle one concept after the other.

Unit Testing is Dynamic Testing

Dynamic testing, which unit testing is a part of, is all about executing code. By doing so, potential defects in the code will manifest as failures, which can therefore be described as visible or in any other way experienceable result of one or more bugs. These failures will then get debugged, the individual bugs will get fixed, and, as a consequence, the quality of the whole software will have been improved.

Validation

Dynamically testing out software is primarily validation, which I would like to explain by simply referring to the following definition:

The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements.
IEEE Std 610.12-1990

The essential question of validation, in the context of software development, is: “Are we building the right product?”—while verification, on the other hand, would ask “Are we building the product right?”.

Static Testing

Even though it has nothing to do with unit testing, I would like to mention this: the opposite of dynamic testing is called static testing, and it does not include any code execution, but reading and (static) analysis only. Static testing is verification.

Unit Testing Requires Full Knowledge of Application Internals

For dynamic testing, there are different testing methods one can apply different testing techniques of. These testing methods are white-box testing (also known as glass box testing), black-box testing, and gray-box testing, which is a combination of the previous two. The major difference between these testing methods is the amount of knowledge about an application’s code one has to have.

Unit testing is applying white-box testing techniques, meaning close examination of procedural level of detail. In order for this to work, the person designing the test is required to have full knowledge of internals.

Let’s assume we are asked to unit test the following (PHP or JavaScript) code:

function answer() {

    return '42';
}

We have a single function that doesn’t do too much: if called, the function returns a hard-coded value. So, in order to test whether or not the function behaves as expected, we, of course, have to know what the expectation is exactly, right? In our case, the expected behavior is that the function returns the string “42”.

Defining Units

Unit testing is about testing units; we got that. But what is a unit?

Unfortunately, this is no question with an easy or absolute answer. In a WordPress context, however, where we have to deal with PHP and JavaScript code, most of the time you will find that a unit represents a single function (or method).

But let’s consider what different classifications of units there are, and which one might make the most sense.

From Characters to Classes

When thinking about units in code, the atomic level is most likely characters. I mean, yeah, if you consider multi-byte characters, you could even split that up into individual bytes (oh, and then there’s bits!), but … let’s just stop short of doing that. 😀 So, characters it is.

On the next level, we can form characters into words. Those words might, for example, be values, or keywords, or names. Stringing together multiple words would then lead to sentences. And if we respect the syntax of our programming language, this would mean expressions (or statements).

Yet one level up, we would have logic: combining words and expressions—some of which are keywords, a few are values, some operators here and there. Program logic is also the first classification of a unit that is both able to get tested and worthy of being tested.

Still considering PHP and JavaScript (i.e., languages that allow for procedural as well as object-oriented programming), the next two levels would be functions and classes.

Reasonable Units

Now, going even higher to namespaces, packages and things like that doesn’t make any sense. We already found that logic is testable, so functions and classes are as well. We should therefore be able to easily define a unit as either logic, or function, or class.

When thinking about logic again, however, we quickly come to the conclusion that, even though it might be testable, it is very often not testable as a unit. Let me try to explain this by means of the following code:

function convert( $number ) {

    if ( $number > 0 ) {
        return $number * 2;
    }

    return $number / 2;
}

In the above function, there is three times logic to be found: one condition, and two mathematical computations. What is important here, however, is that neither of the computations can be targeted (and thus tested) without having the condition involved. Of course, this is obvious. In order to return twice the number passed to the function, the input has to be greater than 0. On the other hand, the second return statement is only reached if the condition evaluated to false. Last but not least: you can also not test the condition alone, for one of the return statements will eventually be reached, which involves one of the two possible computations.

Logic can therefore not be tested completely uncoupled from potential other logic, and thus it cannot be tested as a real unit. All in all, this means that a reasonable unit is either a function or a class. And once again, most of the time, function wins.

Testing in Isolation

One of the most important and yet often overlooked aspects of unit testing is that each and every unit has to be tested in isolation. This means that you only execute the code of the unit you want to test, and no other unit.

Why Test in Isolation?

The reason for this is simple: if your test fails, you cannot know what went wrong where. You execute a function that calls one or more further functions, which again might call yet other functions, and … you have no control, nor knowledge about what happens exactly.

If the only real code that is being executed is the one of your method under test (and, for OOP, maybe other private methods, which are still part of the same class, of course), you know that the reason for a failing unit test is in the method under test itself.

Non-Unit-Test Example

Maybe let’s have a non-unit-test now, to exercise this by means of some example code.

function test_register_taxonomy() {

    $tax = rand_str();

    $this->assertFalse( taxonomy_exists( $tax ) );

    register_taxonomy( $tax, 'post' );

    $this->assertTrue( taxonomy_exists( $tax ) );
    $this->assertFalse( is_taxonomy_hierarchical( $tax ) );

    unset( $GLOBALS['wp_taxonomies'][ $tax ] );
}

The above code is taken from the official WordPress development repository. Interesting, and important, is that the test not only executes the real register_taxonomy() function with all the other real units (i.e., functions) being called by it, but that the test itself even includes calls to other real functions (e.g., taxonomy_exists()).

This is not a unit test! This is some sort of integration or even system test (if WordPress core is completely available and loaded). But still, the test has every right to exist, for unit testing is not everything there is to test your code. What’s important, however, is that there are unit tests for all units involved here, so one can be sure each isolated unit behaves as expected.

Mocking Dependencies to Take Control

So, how does one make sure a unit is tested in isolation? The answer reads easy and yet is not always easy to do: define dummy functions that you have absolute control over.

Depending on the tools you use and/or the literature you read,  you might come across mocks, stubs, spies, fake objects, fake functions, dummy functions, test doubles, and other things. In this post, I will mostly use the term mock (object).

What all of these have in common is that you take care of two things:

• you can execute the real method under test without any errors thrown;
• all other units are only fake ones without real logic, if any.

In languages where this is possible, you can simply (re)define existing functions, methods and whole objects like you want them to be—maybe with some way to revert your changes after the test. JavaScript is such a language. In PHP, on the other hand, you cannot redefine a function, method or class that has already been defined before. This means that you have to take care of the entity that you want to mock not being defined before your test—and also not after, because this would still lead to an error.

For some, maybe even the most, of your unit tests, you cannot just make sure things are available. You also have to take care that mocked units output or return values that, at least, make sense and follow the requirements.

Mocking out of Convenience

But there’s another (good) reason for mocking objects instead of creating real objects: convenience. 🙂

Let’s assume the method under test gets passed an object that is not used in the method itself, but rather passed to some other function (i.e., unit), no matter if as a whole or only one or more properties of it. In this case, you don’t have to mock the object, because it does not directly affect the result (i.e., output or return value) of your method under test. So you could simply take care of the file with the class being (auto)loaded and instantiate a real object.

This might be a problem (or rather: a lot of work), however, if this object has one more dependencies of its own. So, instead of loading multiple files with multiple classes, instantiating multiple real objects with their individual dependencies, maybe providing dummy data for required scalar paramaters, … you could simply mock one object, and be done. Makes sense, right? 😉

Unit Test Examples

The following sections include some unit test examples—with increasing complexity, but still simple enough, I hope. Depending on the situation at hand, various tools and/or packages are used.

All of the following code can be found, fully documented, in a dedicated GitHub repository. If you would like to see unit tests of real-life projects, please refer to WP REST Starter for PHP, and MultilingualPress for JavaScript.

Example 1

In order to start simple, the first example involves a class with a single stupid method only, and there are no dependencies whatsoever.

Carrying out unit testing here requires no more than a (unit) testing framework. For PHP code, this could be atoum, PHPUnit, or SimpleTest, to only name a few. Unit tests for JavaScript code can be done by using, for example, AVA, Jasmine, Jest, Mocha, or tape.

Testee

Assuming we have to unit test the following class—the testee—, we can easily see there is only one possible unit: the answer() method, which we know as a regular function from before.

class Oracle {

    public function answer() {

        return '42';
    }
}

Unit Test with PHPUnit

Since answer() only returns a hard-coded value, testing the method is easy, and can be done by a single test. All we have to do is call the method, and compare what it returns with what we expect it to return (i.e., the string “42”).

So unit testing involves setting an expectation, and then asserting that the actual result matches the expected one. Using PHPUnit, this would look something like the following:

class OracleTest extends PHPUnit_Framework_TestCase {

    public function test_return_expected_answer() {

        $this->assertSame( '42', ( new Oracle() )->answer() );
    }
}

The above code includes a test class, OracleTest, for a single testee, Oracle. This test class contains for all methods of the testee one or more test methods, each targeting another scenario.

For our very simple example of a class with a single method that only returns a hard-coded value, we end up with one test class containing one test method that again includes a single assertion only.

JavaScript Unit Test with tape

Assuming an equivalent ES6 class, the according JavaScript unit test could be like this:

import test from 'tape';
import Oracle from './src/Oracle';

test( ( assert ) => {
    assert.equal( ( new Oracle() ).answer(), '42' );

    assert.end();
} );

The above code uses tape for testing, and it is very similar to the PHP unit test.

Example 2

After a very simple first example, let us continue with a slightly more complex testee: a class with a conditional method, and an external dependency.

Since we only want to test our unit (i.e., the testee’s method), we have to mock the depended-upon class—unless the object is only passed around, meaning it is not used to perform logic itself. Therefore, we (maybe) have to use a mocking framework. For PHP, the most prominent ones would be Mockery, and Prophecy. For JavaScript, I only know of Sinon.JS, which is great, by the way! If you happen to use Jest, you most probably won’t need anything special in order to mock, because Jest comes with decent built-in mocking functionality already.

Testee

And this is the concrete code for our second testee, a fortune teller who will provide you with the answer of their oracle friend only if you pay enough, and say nothing at all otherwise:

class FortuneTeller {

    private $oracle;

    public function __construct( Oracle $oracle ) {

        $this->oracle = $oracle;
    }

    public function answer( $money = 0 ) {

        if ( $money < 5 ) {
            return '';
        }

        return $this->oracle->answer();
    }
}

Unit Test with PHPUnit

Using the built-in mocking functionality of PHPUnit, a unit test could look something like the following:

class FortuneTellerTest extends PHPUnit_Framework_TestCase {

    public function test_return_expected_answer() {

        $answer = 'some answer here';

        $oracle = $this->getMockBuilder( 'Oracle' )
            ->getMock();
        $oracle->method( 'answer' )
            ->willReturn( $answer );

        $testee = new FortuneTeller( $oracle );

        $this->assertSame( '', $testee->answer( 0 ) );
        $this->assertSame( $answer, $testee->answer( 100 ) );
    }
}

Instead of injecting a real oracle, which we don’t have any control over, we create a mock oracle and define the desired behavior (for our test). Reading the code of the fortune teller class, we easily see that there are two possible outcomes: either you don’t get an answer (i.e., the fortune teller returns an empty string), or you get the answer of the oracle (whatever that may be). A very basic unit test would therefore include two scenarios: one time you spend enough money, and the other time you don’t. This is what the above code shows.

A better unit test might also include further scenarios, for example, paying exactly the required amount of money, or a negative amount of money, or maybe even not an integer (or numeric value) at all. Assuming euros as currency where you have one hundred cents making up to one euro, you could also test the closest possible value just below and above the threshold (i.e., 4.99 and 5.01). This kind of test is called boundary-value analysis.

By the way, if you write a PHPUnit test method that includes several assertions that all follow the same structure, you might as well better use a data provider for this. By doing so, you both keep your test method concise, and at the same time separate the test logic from its configuration (e.g., input data, potential mock return values, and expected results).

Unit Test with PHPUnit and Mockery

Another way to mock the oracle is by using Mockery, which I personally like better than PHPUnit’s mocking functionality. Writing the test then is completely along the lines of the PHPUnit-only version. We create a mock object and take control over it (i.e., its answer() method), like so:

class FortuneTellerMockeryTest extends PHPUnit_Framework_TestCase {

    use Mockery\Adapter\Phpunit\MockeryPHPUnitIntegration;

    public function test_return_expected_answer() {

        $answer = 'some answer here';

        $oracle = Mockery::mock( 'Oracle', [
            'answer' => $answer,
        ] );

        $testee = new FortuneTeller( $oracle );

        $this->assertSame( '', $testee->answer( 0 ) );
        $this->assertSame( $answer, $testee->answer( 100 ) );
    }
}

The trait in the above code is just to take care of integrating Mockery with PHPUnit.

JavaScript Unit Test with tape and Sinon.JS

Just like with the oracle class, we assume to also have a fortune teller ES6 class. This could then get tested with the following code:

import test from 'tape';
import sinon from 'sinon';
import FortuneTeller from './src/FortuneTeller';

test( ( assert ) => {
    const answer = 'some answer here';

    const oracle = {
        answer: sinon.stub().returns( answer )
    };

    const testee = new FortuneTeller( oracle );

    assert.equal( testee.answer( 0 ), '' );

    assert.equal( testee.answer( 100 ), answer );

    assert.end();
} );

For our JavaScript test, it is enough if we define the oracle as an object literal with the required function set up like we want it to behave; in the terminology used by Sinon.JS, we thereby create a stub. The rest is pretty similar to what we did in the PHP unit test.

Example 3

For the third example, we simply take the first one again, and extend it a little by making it aware of WordPress action and filter hooks.

To make life easier, we might want to use some WordPress mocking framework, for example, Brain Monkey, or WP_Mock.

Testee

The new hook-aware oracle is similar to our old oracle friend. But see for yourself:

class HookAwareOracle {

    const ACTION = 'give_answer';

    const FILTER = 'the_answer';

    public function answer() {

        do_action( self::ACTION );

        return (string) apply_filters( self::FILTER, '42' );
    }
}

What makes the new oracle special is firing a WordPress action as soon as the answer() method is called, and passing the answer through a dedicated WordPress filter before returning it.

Unit Test with PHPUnit and Brain Monkey

The according unit test includes what the one in our first example already has: an assertion of the result we obtain from the method. However, there is (or better: we make) a difference: since the answer is filterable, we might or might not get the original answer of the oracle. In the latter case this means that it got filtered (i.e., altered).

We therefore write two separate test methods, one of which handles the case where there is no filter callback added, and the other where there is. In both methods, we assert that the action got fired, and we also assert that the according set of filters got applied, no matter if there are any or not. The difference regarding the filter is that, in the second method, we are not only interested in the filters being applied, but also define a single filter ourselves: one that returns a custom and yet pre-defined string that we can compare the actual result against.

class HookAwareOracleTest extends PHPUnit_Framework_TestCase {

    // Set up and tear down Brain Monkey...

    public function test_return_unfiltered_answer() {

        $this->assertSame( '42', ( new HookAwareOracle() )->answer() );

        $this->assertSame( 1, did_action( HookAwareOracle::ACTION ) );

        $this->assertSame( 1, Monkey::filters()->applied( HookAwareOracle::FILTER ) );
    }

    public function test_return_filtered_answer() {

        $answer = 'some answer here';

        Monkey\Filters::expectApplied( HookAwareOracle::FILTER )
            ->once()
            ->andReturn( $answer );

        $this->assertSame( $answer, ( new HookAwareOracle() )->answer() );

        $this->assertSame( 1, did_action( HookAwareOracle::ACTION ) );
    }
}

Writing even more test methods for two, three etc. filter callbacks being registered doesn’t do any good here, because this would actually not test our method, but rather whether or not WordPress correctly handles adding and applying multiple filters.

What about JavaScript?

Currently, there is no JavaScript API to realize WordPress actions and filters, so we cannot set up a similar example by using WordPress core functions only.

However, since there is an according ticket on Trac, which is currently being worked on, this might be possible one day, maybe.

Example 4

In the fourth and last example, there is a WordPress function being used. Since this function is a separate unit, we don’t want it to interfere with our unit, and thus we have to patch it. This can be done with Patchwork, or Brain Monkey, which we know already. Internally, Brain Monkey uses Patchwork itself.

Testee

The code is simple. We have a new oracle that returns a random number as answer, in a full sentence, translated according to the current locale. This is done by using WordPress’s __() function, which we now have to patch.

Here is the translating oracle:

class TranslatingOracle {

    public function answer() {

        return sprintf(
            __( 'The answer is: %d.', 'some-text-domain-here' ),
            mt_rand()
        );
    }
}

Unit Test with PHPUnit and Patchwork

Basically, what we want with our test is to make sure we get an answer that contains a valid random number. And we do not want to have the real WordPress function be executed. Since we know how sprintf() works, we can use this information, and make the __() function return the string %d, which will then result in only the random number being returned as string.

Using Patchwork, the unit test would look something like this:

// Make sure vendor/antecedent/patchwork/Patchwork.php is included early!

class TranslatingOraclePatchworkTest extends PHPUnit_Framework_TestCase {

    // Tear down Patchwork...

    public function test_return_expected_answer() {

        Patchwork\replace( '__', function () {

            return '%d';
        } );

        $this->assertRegExp( '/^\d+$/', ( new TranslatingOracle() )->answer() );
    }
}

As you can see, we are not even interested if any translation is being performed, nor do we check the given text domain. These facts simply do not contribute to the abstract definition of what the method is supposed to do.

Unit Test with PHPUnit and Brain Monkey

When using Brain Monkey, the unit test is very similar to the one before. We take care of the currently undefined WordPress function so the real code can safely be executed without any errors.

class TranslatingOracleBrainMonkeyTest extends PHPUnit_Framework_TestCase {

    // Set up and tear down Brain Monkey...

    public function test_return_expected_answer() {

        Monkey\Functions::expect( '__' )
            ->once()
            ->andReturn( '%d' );

        $this->assertRegExp( '/^\d+$/', ( new TranslatingOracle() )->answer() );
    }
}

However, there is a difference to the Patchwork test: with Brain Monkey, we are also able to set expectations. Since the class represents a translating oracle, translation is an integral part of the method. We therefore not only allow for the __() function to get called, we also expect it to get called exactly once. Thus, this unit test is more powerful and also better than the Patchwork version.

JavaScript Version

To make things a little more interesting (and easier, for there is no I18n functionality in JavaScript, yet 😉 ), the JavaScript code is somewhat different from PHP, but follows a similar concept: there is a function being used that is defined in a WordPress JavaScript file. This is it:

class ThankfulOracle {
    answer( money ) {
        showNotice.note( `Thanks for "${Number( money )}", buddy.` );

        return Math.random();
    }
}

The code of the according unit test uses, again, tape and Sinon.JS, and looks like so:

import test from 'tape';
import sinon from 'sinon';
import ThankfulOracle from './src/ThankfulOracle';

test( ( assert ) => {
    global.showNotice = {
        note: sinon.spy()
    };

    const money = 42;

    assert.equal( typeof ( new ThankfulOracle() ).answer( money ), 'number' );

    assert.equal( global.showNotice.note.callCount, 1 );

    assert.equal( global.showNotice.note.calledWithMatch( new RegExp( `"${Number( money )}"` ) ), true );

    delete global.showNotice;

    assert.end();
} );

In the above test, we mock the WordPress JavaScript function (i.e., showNotice.note()). In fact, we create a spy only, because the method does not return anything (that might be used in the logic of the answer() method itself. We are only interested in whether or not the method got called (once, that is), and if it got passed the correct amount of money. For the latter part, we could check if it was called with the complete string. However, the string itself is not important, only the fact that the money is included in it. If we were to tailor the test to the exact current (!) string, we would have to adapt the test as soon as we change only a single character in that string. Otherwise the test would fail. And this is just nonsense.

Since the oracle returns a random string, we cannot compare the result with a concrete value. But it also doesn’t matter, because the oracle behaves correctly as long as we get a number. So this is what we check then: the type of the return value.

Lastly, we undo the changes we made to the showNotice object/namespace. Because it is global, it would still exist after our test method, and might lead to unexpected results in other tests. This is something very important that we always have to keep in mind.

Unit Tests for Production Code

When it comes to real-life projects, neither the according code, nor the final unit tests will look like what we have seen before. The reason is simple: the example code and thus the resulting unit tests were designed to be simple, and only included one or two (new) aspects of unit testing.

In production code, you will most likely have a combination of what we before handled in two or more examples. However, if you understand the constructed example code and their individual unit test, then you are also capable of tackling by far more complex real-life code. 🙂

Writing Good Unit Tests

Now that we have seen a few and therefore should be able to write simple unit tests ourselves, the big question is how to do this well. The following section is mostly a summary of this great post by Eric Elliott.

Every Unit Test Should Answer Five Questions

While there is a lot more to writing good unit tests, providing answers to the following five questions is a huge step in the right direction:

1. What are you testing?
2. What should it do?
3. What is the actual result (i.e., output, or return value)?
4. What is the expected result?
5. How can the test be reproduced?

The answers for the first four questions can be found in your unit tests in variable or function names, or messages (i.e., strings). The fifth question is answered by looking at how the “actual” result is generated.

Unit Test Templates

Putting what we learned in the previous section into a unit test will result in a helpful template to start with.

For PHP, this could look something like this:

class WhatClassTest extends PHPUnit_Framework_TestCase {

    public function test_what_method() {

        $actual = 'What is the actual result?';

        $expected = 'What is the expected result?';

        $this->assertSame( $expected, $actual, 'What should it do?' );
    }
}

The JavaScript version might be similar to the following:

import test from 'tape';

test( 'What are you testing?', ( assert ) => {
    const actual = 'What is the actual result?';
    const expected = 'What is the expected result?';

    assert.equal( actual, expected, 'What should it do?' );

    assert.end();
} );

Summary

OK, that was surely a lot. Let’s try to boil it down to our personal unit test essentials now.

Unit testing…

• …aims at finding defects;
• …involves execution of code;
• …means testing of individual units in isolation;
• …requires full knowledge of application internals;
• …determines if the software satisfies the requirements.

Alrighty, that looks like a manageable list to me. 🙂

What Next?

Now that we have a common basic understanding of unit testing, the next step would be to actually discuss what makes code unit testable. I will do this in a series of, most probably, three or four posts. So stay tuned. 🙂

If you have a question, feel the need to note one thing or another, or want to make sure I include some specific aspect in these future posts, … well, there is this thing called comments. 🙂 Please, let me know!

Thanks to Alain Schlesser and Giuseppe Mazzapica for reading an early draft of this post, and providing valuable feedback.