summaryrefslogtreecommitdiffstatshomepage
path: root/3rdparty/catch/docs/tutorial.md
blob: 7392459262e131e1b5dae0edf947d754d5d6b984 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
# Getting Catch

The simplest way to get Catch is to download the latest [single header version](https://raw.githubusercontent.com/philsquared/Catch/master/single_include/catch.hpp). The single header is generated by merging a set of individual headers but it is still just normal source code in a header file.

The full source for Catch, including test projects, documentation, and other things, is hosted on GitHub. [http://catch-lib.net](http://catch-lib.net) will redirect you there.


## Where to put it?

Catch is header only. All you need to do is drop the file(s) somewhere reachable from your project - either in some central location you can set your header search path to find, or directly into your project tree itself! This is a particularly good option for other Open-Source projects that want to use Catch for their test suite. See [this blog entry for more on that](http://www.levelofindirection.com/journal/2011/5/27/unit-testing-in-c-and-objective-c-just-got-ridiculously-easi.html). 

The rest of this tutorial will assume that the Catch single-include header (or the include folder) is available unqualified - but you may need to prefix it with a folder name if necessary.

# Writing tests

Let's start with a really simple example. Say you have written a function to calculate factorials and now you want to test it (let's leave aside TDD for now). 

```c++
unsigned int Factorial( unsigned int number ) {
    return number <= 1 ? number : Factorial(number-1)*number;
}
```

To keep things simple we'll put everything in a single file (<a href="#scaling-up">see later for more on how to structure your test files</a>)

```c++
#define CATCH_CONFIG_MAIN  // This tells Catch to provide a main() - only do this in one cpp file
#include "catch.hpp"

unsigned int Factorial( unsigned int number ) {
    return number <= 1 ? number : Factorial(number-1)*number;
}

TEST_CASE( "Factorials are computed", "[factorial]" ) {
    REQUIRE( Factorial(1) == 1 );
    REQUIRE( Factorial(2) == 2 );
    REQUIRE( Factorial(3) == 6 );
    REQUIRE( Factorial(10) == 3628800 );
}
```

This will compile to a complete executable which responds to [command line arguments](command-line.md). If you just run it with no arguments it will execute all test cases (in this case there is just one), report any failures, report a summary of how many tests passed and failed and return the number of failed tests (useful for if you just want a yes/ no answer to: "did it work").

If you run this as written it will pass. Everything is good. Right?
Well, there is still a bug here. In fact the first version of this tutorial I posted here genuinely had the bug in! So it's not completely contrived (thanks to Daryle Walker (```@CTMacUser```) for pointing this out).

What is the bug? Well what is the factorial of zero?
[The factorial of zero is one](http://mathforum.org/library/drmath/view/57128.html) - which is just one of those things you have to know (and remember!).

Let's add that to the test case:

```c++
TEST_CASE( "Factorials are computed", "[factorial]" ) {
    REQUIRE( Factorial(0) == 1 );
    REQUIRE( Factorial(1) == 1 );
    REQUIRE( Factorial(2) == 2 );
    REQUIRE( Factorial(3) == 6 );
    REQUIRE( Factorial(10) == 3628800 );
}
```

Now we get a failure - something like:

```
Example.cpp:9: FAILED:
  REQUIRE( Factorial(0) == 1 )
with expansion:
  0 == 1
```

Note that we get the actual return value of Factorial(0) printed for us (0) - even though we used a natural expression with the == operator. That let's us immediately see what the problem is.

Let's change the factorial function to:

```c++
unsigned int Factorial( unsigned int number ) {
  return number > 1 ? Factorial(number-1)*number : 1;
}
```

Now all the tests pass.

Of course there are still more issues to do deal with. For example we'll hit problems when the return value starts to exceed the range of an unsigned int. With factorials that can happen quite quickly. You might want to add tests for such cases and decide how to handle them. We'll stop short of doing that here.

## What did we do here?

Although this was a simple test it's been enough to demonstrate a few things about how Catch is used. Let's take moment to consider those before we move on.

1. All we did was ```#define``` one identifier and ```#include``` one header and we got everything - even an implementation of ```main()``` that will [respond to command line arguments](command-line.md). You can only use that ```#define``` in one implementation file, for (hopefully) obvious reasons. Once you have more than one file with unit tests in you'll just ```#include "catch.hpp"``` and go. Usually it's a good idea to have a dedicated implementation file that just has ```#define CATCH_CONFIG_MAIN``` and ```#include "catch.hpp"```. You can also provide your own implementation of main and drive Catch yourself (see [Supplying-your-own-main()](own-main.md)).
2. We introduce test cases with the ```TEST_CASE``` macro. This macro takes one or two arguments - a free form test name and, optionally, one or more tags (for more see <a href="#test-cases-and-sections">Test cases and Sections</a>, ). The test name must be unique. You can run sets of tests by specifying a wildcarded test name or a tag expression. See the [command line docs](command-line.md) for more information on running tests.
3. The name and tags arguments are just strings. We haven't had to declare a function or method - or explicitly register the test case anywhere. Behind the scenes a function with a generated name is defined for you, and automatically registered using static registry classes. By abstracting the function name away we can name our tests without the constraints of identifier names.
4. We write our individual test assertions using the ```REQUIRE``` macro. Rather than a separate macro for each type of condition we express the condition naturally using C/C++ syntax. Behind the scenes a simple set of expression templates captures the left-hand-side and right-hand-side of the expression so we can display the values in our test report. As we'll see later there _are_ other assertion macros - but because of this technique the number of them is drastically reduced.

<a id="test-cases-and-sections"></a>
## Test cases and sections

Most test frameworks have a class-based fixture mechanism. That is, test cases map to methods on a class and common setup and teardown can be performed in ```setup()``` and ```teardown()``` methods (or constructor/ destructor in languages, like C++, that support deterministic destruction).

While Catch fully supports this way of working there are a few problems with the approach. In particular the way your code must be split up, and the blunt granularity of it, may cause problems. You can only have one setup/ teardown pair across a set of methods, but sometimes you want slightly different setup in each method, or you may even want several levels of setup (a concept which we will clarify later on in this tutorial). It was <a href="http://jamesnewkirk.typepad.com/posts/2007/09/why-you-should-.html">problems like these</a> that led James Newkirk, who led the team that built NUnit, to start again from scratch and <a href="http://jamesnewkirk.typepad.com/posts/2007/09/announcing-xuni.html">build xUnit</a>).

Catch takes a different approach (to both NUnit and xUnit) that is a more natural fit for C++ and the C family of languages. This is best explained through an example:

```c++
TEST_CASE( "vectors can be sized and resized", "[vector]" ) {

    std::vector<int> v( 5 );
    
    REQUIRE( v.size() == 5 );
    REQUIRE( v.capacity() >= 5 );
    
    SECTION( "resizing bigger changes size and capacity" ) {
        v.resize( 10 );
        
        REQUIRE( v.size() == 10 );
        REQUIRE( v.capacity() >= 10 );
    }
    SECTION( "resizing smaller changes size but not capacity" ) {
        v.resize( 0 );
        
        REQUIRE( v.size() == 0 );
        REQUIRE( v.capacity() >= 5 );
    }
    SECTION( "reserving bigger changes capacity but not size" ) {
        v.reserve( 10 );
        
        REQUIRE( v.size() == 5 );
        REQUIRE( v.capacity() >= 10 );
    }
    SECTION( "reserving smaller does not change size or capacity" ) {
        v.reserve( 0 );
        
        REQUIRE( v.size() == 5 );
        REQUIRE( v.capacity() >= 5 );
    }
}
```

For each ```SECTION``` the ```TEST_CASE``` is executed from the start - so as we enter each section we know that size is 5 and capacity is at least 5. We enforced those requirements with the ```REQUIRE```s at the top level so we can be confident in them.
This works because the ```SECTION``` macro contains an if statement that calls back into Catch to see if the section should be executed. One leaf section is executed on each run through a ```TEST_CASE```. The other sections are skipped. Next time through the next section is executed, and so on until no new sections are encountered.

So far so good - this is already an improvement on the setup/teardown approach because now we see our setup code inline and use the stack.

The power of sections really shows, however, when we need to execute a sequence of, checked, operations. Continuing the vector example, we might want to verify that attempting to reserve a capacity smaller than the current capacity of the vector changes nothing. We can do that, naturally, like so:

```c++
    SECTION( "reserving bigger changes capacity but not size" ) {
        v.reserve( 10 );
        
        REQUIRE( v.size() == 5 );
        REQUIRE( v.capacity() >= 10 );
    
        SECTION( "reserving smaller again does not change capacity" ) {
            v.reserve( 7 );
            
            REQUIRE( v.capacity() >= 10 );
        }
    }
```

Sections can be nested to an arbitrary depth (limited only by your stack size). Each leaf section (i.e. a section that contains no nested sections) will be executed exactly once, on a separate path of execution from any other leaf section (so no leaf section can interfere with another). A failure in a parent section will prevent nested sections from running - but then that's the idea.

## BDD-Style

If you name your test cases and sections appropriately you can achieve a BDD-style specification structure. This became such a useful way of working that first class support has been added to Catch. Scenarios can be specified using ```SCENARIO```, ```GIVEN```, ```WHEN``` and ```THEN``` macros, which map on to ```TEST_CASE```s and ```SECTION```s, respectively. For more details see [Test cases and sections](test-cases-and-sections.md).

The vector example can be adjusted to use these macros like so:

```c++
SCENARIO( "vectors can be sized and resized", "[vector]" ) {

    GIVEN( "A vector with some items" ) {
        std::vector<int> v( 5 );
        
        REQUIRE( v.size() == 5 );
        REQUIRE( v.capacity() >= 5 );
        
        WHEN( "the size is increased" ) {
            v.resize( 10 );
            
            THEN( "the size and capacity change" ) {
                REQUIRE( v.size() == 10 );
                REQUIRE( v.capacity() >= 10 );
            }
        }
        WHEN( "the size is reduced" ) {
            v.resize( 0 );
            
            THEN( "the size changes but not capacity" ) {
                REQUIRE( v.size() == 0 );
                REQUIRE( v.capacity() >= 5 );
            }
        }
        WHEN( "more capacity is reserved" ) {
            v.reserve( 10 );
            
            THEN( "the capacity changes but not the size" ) {
                REQUIRE( v.size() == 5 );
                REQUIRE( v.capacity() >= 10 );
            }
        }
        WHEN( "less capacity is reserved" ) {
            v.reserve( 0 );
            
            THEN( "neither size nor capacity are changed" ) {
                REQUIRE( v.size() == 5 );
                REQUIRE( v.capacity() >= 5 );
            }
        }
    }
}
```

Conveniently, these tests will be reported as follows when run:

```
Scenario: vectors can be sized and resized
     Given: A vector with some items
      When: more capacity is reserved
      Then: the capacity changes but not the size
```

<a id="scaling-up"></a>
## Scaling up

To keep the tutorial simple we put all our code in a single file. This is fine to get started - and makes jumping into Catch even quicker and easier. As you write more real-world tests, though, this is not really the best approach.

The requirement is that the following block of code ([or equivalent](own-main.md)):

```c++
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
```

appears in _exactly one_ source file. Use as many additional cpp files (or whatever you call your implementation files) as you need for your tests, partitioned however makes most sense for your way of working. Each additional file need only ```#include "catch.hpp"``` - do not repeat the ```#define```!

In fact it is usually a good idea to put the block with the ```#define``` [in its own source file](slow-compiles.md).

Do not write your tests in header files!


## Next steps

This has been a brief introduction to get you up and running with Catch, and to point out some of the key differences between Catch and other frameworks you may already be familiar with. This will get you going quite far already and you are now in a position to dive in and write some tests.

Of course there is more to learn - most of which you should be able to page-fault in as you go. Please see the ever-growing [Reference section](Readme.md) for what's available.

---

[Home](Readme.md)
Vas Crabb15 months mame0261commit ca50094e8d... Vas Crabb17 months mame0260commit 0a7f1fe9cf... Vas Crabb18 months mame0259commit 4ff20056c3... Vas Crabb19 months mame0258commit 2e0aa82350... Vas Crabb20 months mame0257commit f811a66c53... Vas Crabb21 months mame0256commit b41370db02... Vas Crabb22 months mame0255commit c6650dc072... Vas Crabb23 months mame0254commit bfa8d724a0... Vas Crabb2 years mame0253commit b6d9756c5e... Vas Crabb2 years mame0252commit fb98822c34... Vas Crabb2 years mame0251commit 34e6ec1ef8... Vas Crabb2 years mame0250commit b7cbe74c4b... Vas Crabb2 years mame0249commit 91c5b9ecea... Vas Crabb3 years mame0248commit 2d3d0deec8... Vas Crabb3 years mame0247commit fa2d36c634... Vas Crabb3 years mame0246commit 205b03897c... Vas Crabb3 years mame0245commit 5d31f0fc97... Vas Crabb3 years mame0244commit bcf77373a5... Vas Crabb3 years mame0243commit addbb8ab40... Vas Crabb3 years mame0242commit e8166b5274... Vas Crabb3 years mame0241commit 31f001e501... Vas Crabb3 years mame0240commit f0ab44fe1c... Vas Crabb3 years mame0239commit 80bcaea1ed... Vas Crabb3 years mame0238commit fb21b78904... Vas Crabb3 years mame0237commit 34d8357465... Vas Crabb4 years mame0236commit 5e865af540... Vas Crabb4 years mame0235commit ec9ba6fa76... Vas Crabb4 years mame0234commit 2633c19a68... Vas Crabb4 years mame0233commit 05d0cf61e7... Vas Crabb4 years mame0232commit 2b0f01bc3a... Vas Crabb4 years mame0231commit 1f22113661... Vas Crabb4 years mame0230commit 943c06cba0... Vas Crabb4 years mame0229commit 4322eaae9d... Vas Crabb4 years mame0228commit 140f446933... Vas Crabb4 years mame0227commit d85735634c... Vas Crabb4 years mame0226commit 3c56452b07... Vas Crabb5 years mame0225commit 5a1fd0cc17... Vas Crabb5 years mame0224commit 5892c78a15... Vas Crabb5 years mame0223commit c55a261d26... Vas Crabb5 years mame0222commit 6d50d60a43... Vas Crabb5 years mame0221commit e8a0e0469b... Vas Crabb5 years mame0220commit c5c5723b9d... Vas Crabb5 years mame0219commit 221f006442... Vas Crabb5 years mame0218commit 0e2a252d30... Vas Crabb5 years mame0217commit 13997a8f31... Vas Crabb5 years mame0216commit b8b7c7e232... Vas Crabb5 years mame0215commit e9ef4808dd... Vas Crabb6 years mame0214commit 24d07a12d7... Vas Crabb6 years mame0213commit f7172322a2... Vas Crabb6 years mame0212commit 1182bd9325... Vas Crabb6 years mame0211commit 1b969a8acb... Vas Crabb6 years mame0210commit ad45c9c609... Vas Crabb6 years mame0209commit 2b317bf296... Vas Crabb6 years mame0208commit 9483624864...