Subversion-Projekte lars-tiefland.prado

Revision

Details | Letzte Änderung | Log anzeigen | RSS feed

Revision Autor Zeilennr. Zeile
1 lars 1
<html>
2
<head>
3
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
4
<title>SimpleTest for PHP test runner and display documentation</title>
5
<link rel="stylesheet" type="text/css" href="docs.css" title="Styles">
6
</head>
7
<body>
8
<div class="menu_back">
9
<div class="menu">
10
<h2>
11
<a href="index.html">SimpleTest</a>
12
</h2>
13
<ul>
14
<li>
15
<a href="overview.html">Overview</a>
16
</li>
17
<li>
18
<a href="unit_test_documentation.html">Unit tester</a>
19
</li>
20
<li>
21
<a href="group_test_documentation.html">Group tests</a>
22
</li>
23
<li>
24
<a href="mock_objects_documentation.html">Mock objects</a>
25
</li>
26
<li>
27
<a href="partial_mocks_documentation.html">Partial mocks</a>
28
</li>
29
<li>
30
<span class="chosen">Reporting</span>
31
</li>
32
<li>
33
<a href="expectation_documentation.html">Expectations</a>
34
</li>
35
<li>
36
<a href="web_tester_documentation.html">Web tester</a>
37
</li>
38
<li>
39
<a href="form_testing_documentation.html">Testing forms</a>
40
</li>
41
<li>
42
<a href="authentication_documentation.html">Authentication</a>
43
</li>
44
<li>
45
<a href="browser_documentation.html">Scriptable browser</a>
46
</li>
47
</ul>
48
</div>
49
</div>
50
<h1>Test reporter documentation</h1>
51
<div class="content">
52
 
53
            <p>
54
                SimpleTest pretty much follows the MVC pattern
55
                (Model-View-Controller).
56
                The reporter classes are the view and the model is your
57
                test cases and their hiearchy.
58
                The controller is mostly hidden from the user of
59
                SimpleTest unless you want to change how the test cases
60
                are actually run, in which case it is possible to
61
                override the runner objects from within the test case.
62
                As usual with MVC, the controller is mostly undefined
63
                and there are other places to control the test run.
64
            </p>
65
 
66
        <p>
67
<a class="target" name="html">
68
<h2>Reporting results in HTML</h2>
69
</a>
70
</p>
71
            <p>
72
                The default test display is minimal in the extreme.
73
                It reports success and failure with the conventional red and
74
                green bars and shows a breadcrumb trail of test groups
75
                for every failed assertion.
76
                Here's a fail...
77
                <div class="demo">
78
                    <h1>File test</h1>
79
                    <span class="fail">Fail</span>: createnewfile-&gt;True assertion failed.<br>
80
                    <div style="padding: 8px; margin-top: 1em; background-color: red; color: white;">1/1 test cases complete.
81
                    <strong>0</strong> passes, <strong>1</strong> fails and <strong>0</strong> exceptions.</div>
82
                </div>
83
                And here all tests passed...
84
                <div class="demo">
85
                    <h1>File test</h1>
86
                    <div style="padding: 8px; margin-top: 1em; background-color: green; color: white;">1/1 test cases complete.
87
                    <strong>1</strong> passes, <strong>0</strong> fails and <strong>0</strong> exceptions.</div>
88
                </div>
89
                The good news is that there are several points in the display
90
                hiearchy for subclassing.
91
            </p>
92
            <p>
93
                For web page based displays there is the
94
                <span class="new_code">HtmlReporter</span> class with the following
95
                signature...
96
<pre>
97
class HtmlReporter extends SimpleReporter {
98
    public HtmlReporter($encoding) { ... }
99
    public makeDry(boolean $is_dry) { ... }
100
    public void paintHeader(string $test_name) { ... }
101
    public void sendNoCacheHeaders() { ... }
102
    public void paintFooter(string $test_name) { ... }
103
    public void paintGroupStart(string $test_name, integer $size) { ... }
104
    public void paintGroupEnd(string $test_name) { ... }
105
    public void paintCaseStart(string $test_name) { ... }
106
    public void paintCaseEnd(string $test_name) { ... }
107
    public void paintMethodStart(string $test_name) { ... }
108
    public void paintMethodEnd(string $test_name) { ... }
109
    public void paintFail(string $message) { ... }
110
    public void paintPass(string $message) { ... }
111
    public void paintError(string $message) { ... }
112
    public void paintException(string $message) { ... }
113
    public void paintMessage(string $message) { ... }
114
    public void paintFormattedMessage(string $message) { ... }
115
    protected string _getCss() { ... }
116
    public array getTestList() { ... }
117
    public integer getPassCount() { ... }
118
    public integer getFailCount() { ... }
119
    public integer getExceptionCount() { ... }
120
    public integer getTestCaseCount() { ... }
121
    public integer getTestCaseProgress() { ... }
122
}
123
</pre>
124
                Here is what some of these methods mean. First the display methods
125
                that you will probably want to override...
126
                <ul class="api">
127
                    <li>
128
                        <span class="new_code">HtmlReporter(string $encoding)</span>
129
<br>
130
                        is the constructor.
131
                        Note that the unit test sets up the link to the display
132
                        rather than the other way around.
133
                        The display is a mostly passive receiver of test events.
134
                        This allows easy adaption of the display for other test
135
                        systems beside unit tests, such as monitoring servers.
136
                        The encoding is the character encoding you wish to
137
                        display the test output in.
138
                        In order to correctly render debug output when
139
                        using the web tester, this should match the encoding
140
                        of the site you are trying to test.
141
                        The available character set strings are described in
142
                        the PHP <a href="http://www.php.net/manual/en/function.htmlentities.php">html_entities()</a>
143
                        function.
144
                    </li>
145
                    <li>
146
                        <span class="new_code">void paintHeader(string $test_name)</span>
147
<br>
148
                        is called once at the very start of the test when the first
149
                        start event arrives.
150
                        The first start event is usually delivered by the top level group
151
                        test and so this is where <span class="new_code">$test_name</span>
152
                        comes from.
153
                        It paints the page titles, CSS, body tag, etc.
154
                        It returns nothing (<span class="new_code">void</span>).
155
                    </li>
156
                    <li>
157
                        <span class="new_code">void paintFooter(string $test_name)</span>
158
<br>
159
                        Called at the very end of the test to close any tags opened
160
                        by the page header.
161
                        By default it also displays the red/green bar and the final
162
                        count of results.
163
                        Actually the end of the test happens when a test end event
164
                        comes in with the same name as the one that started it all
165
                        at the same level.
166
                        The tests nest you see.
167
                        Closing the last test finishes the display.
168
                    </li>
169
                    <li>
170
                        <span class="new_code">void paintMethodStart(string $test_name)</span>
171
<br>
172
                        is called at the start of each test method.
173
                        The name normally comes from method name.
174
                        The other test start events behave the same way except
175
                        that the group test one tells the reporter how large
176
                        it is in number of held test cases.
177
                        This is so that the reporter can display a progress bar
178
                        as the runner churns through the test cases.
179
                    </li>
180
                    <li>
181
                        <span class="new_code">void paintMethodEnd(string $test_name)</span>
182
<br>
183
                        backs out of the test started with the same name.
184
                    </li>
185
                    <li>
186
                        <span class="new_code">void paintFail(string $message)</span>
187
<br>
188
                        paints a failure.
189
                        By default it just displays the word fail, a breadcrumbs trail
190
                        showing the current test nesting and the message issued by
191
                        the assertion.
192
                    </li>
193
                    <li>
194
                        <span class="new_code">void paintPass(string $message)</span>
195
<br>
196
                        by default does nothing.
197
                    </li>
198
                    <li>
199
                        <span class="new_code">string _getCss()</span>
200
<br>
201
                        Returns the CSS styles as a string for the page header
202
                        method.
203
                        Additional styles have to be appended here if you are
204
                        not overriding the page header.
205
                        You will want to use this method in an overriden page header
206
                        if you want to include the original CSS.
207
                    </li>
208
                </ul>
209
                There are also some accessors to get information on the current
210
                state of the test suite.
211
                Use these to enrich the display...
212
                <ul class="api">
213
                    <li>
214
                        <span class="new_code">array getTestList()</span>
215
<br>
216
                        is the first convenience method for subclasses.
217
                        Lists the current nesting of the tests as a list
218
                        of test names.
219
                        The first, most deeply nested test, is first in the
220
                        list and the current test method will be last.
221
                    </li>
222
                    <li>
223
                        <span class="new_code">integer getPassCount()</span>
224
<br>
225
                        returns the number of passes chalked up so far.
226
                        Needed for the display at the end.
227
                    </li>
228
                    <li>
229
                        <span class="new_code">integer getFailCount()</span>
230
<br>
231
                        is likewise the number of fails so far.
232
                    </li>
233
                    <li>
234
                        <span class="new_code">integer getExceptionCount()</span>
235
<br>
236
                        is likewise the number of errors so far.
237
                    </li>
238
                    <li>
239
                        <span class="new_code">integer getTestCaseCount()</span>
240
<br>
241
                        is the total number of test cases in the test run.
242
                        This includes the grouping tests themselves.
243
                    </li>
244
                    <li>
245
                        <span class="new_code">integer getTestCaseProgress()</span>
246
<br>
247
                        is the number of test cases completed so far.
248
                    </li>
249
                </ul>
250
                One simple modification is to get the HtmlReporter to display
251
                the passes as well as the failures and errors...
252
<pre>
253
<strong>class ShowPasses extends HtmlReporter {
254
 
255
    function paintPass($message) {
256
        parent::paintPass($message);
257
        print "&amp;&lt;span class=\"pass\"&gt;Pass&lt;/span&gt;: ";
258
        $breadcrumb = $this-&gt;getTestList();
259
        array_shift($breadcrumb);
260
        print implode("-&amp;gt;", $breadcrumb);
261
        print "-&amp;gt;$message&lt;br /&gt;\n";
262
    }
263
 
264
    function _getCss() {
265
        return parent::_getCss() . ' .pass { color: green; }';
266
    }
267
}</strong>
268
</pre>
269
            </p>
270
            <p>
271
                One method that was glossed over was the <span class="new_code">makeDry()</span>
272
                method.
273
                If you run this method, with no parameters, on the reporter
274
                before the test suite is run no actual test methods
275
                will be called.
276
                You will still get the events of entering and leaving the
277
                test methods and test cases, but no passes or failures etc,
278
                because the test code will not actually be executed.
279
            </p>
280
            <p>
281
                The reason for this is to allow for more sophistcated
282
                GUI displays that allow the selection of individual test
283
                cases.
284
                In order to build a list of possible tests they need a
285
                report on the test structure for drawing, say a tree view
286
                of the test suite.
287
                With a reporter set to dry run that just sends drawing events
288
                this is easily accomplished.
289
            </p>
290
 
291
        <p>
292
<a class="target" name="other">
293
<h2>Extending the reporter</h2>
294
</a>
295
</p>
296
            <p>
297
                Rather than simply modifying the existing display, you might want to
298
                produce a whole new HTML look, or even generate text or XML.
299
                Rather than override every method in
300
                <span class="new_code">HtmlReporter</span> we can take one
301
                step up the class hiearchy to <span class="new_code">SimpleReporter</span>
302
                in the <em>simple_test.php</em> source file.
303
            </p>
304
            <p>
305
                A do nothing display, a blank canvas for your own creation, would
306
                be...
307
<pre>
308
<strong>require_once('simpletest/simple_test.php');</strong>
309
 
310
class MyDisplay extends SimpleReporter {<strong>
311
    </strong>
312
    function paintHeader($test_name) {
313
    }
314
 
315
    function paintFooter($test_name) {
316
    }
317
 
318
    function paintStart($test_name, $size) {<strong>
319
        parent::paintStart($test_name, $size);</strong>
320
    }
321
 
322
    function paintEnd($test_name, $size) {<strong>
323
        parent::paintEnd($test_name, $size);</strong>
324
    }
325
 
326
    function paintPass($message) {<strong>
327
        parent::paintPass($message);</strong>
328
    }
329
 
330
    function paintFail($message) {<strong>
331
        parent::paintFail($message);</strong>
332
    }
333
}
334
</pre>
335
                No output would come from this class until you add it.
336
            </p>
337
 
338
        <p>
339
<a class="target" name="cli">
340
<h2>The command line reporter</h2>
341
</a>
342
</p>
343
            <p>
344
                SimpleTest also ships with a minimal command line reporter.
345
                The interface mimics JUnit to some extent, but paints the
346
                failure messages as they arrive.
347
                To use the command line reporter simply substitute it
348
                for the HTML version...
349
<pre>
350
&lt;?php
351
    require_once('simpletest/unit_tester.php');
352
    require_once('simpletest/reporter.php');
353
 
354
    $test = &amp;new GroupTest('File test');
355
    $test-&gt;addTestFile('tests/file_test.php');
356
    $test-&gt;run(<strong>new TextReporter()</strong>);
357
?&gt;
358
</pre>
359
                Then invoke the test suite from the command line...
360
<pre class="shell">
361
php file_test.php
362
</pre>
363
                You will need the command line version of PHP installed
364
                of course.
365
                A passing test suite looks like this...
366
<pre class="shell">
367
File test
368
OK
369
Test cases run: 1/1, Failures: 0, Exceptions: 0
370
</pre>
371
                A failure triggers a display like this...
372
<pre class="shell">
373
File test
374
1) True assertion failed.
375
	in createnewfile
376
FAILURES!!!
377
Test cases run: 1/1, Failures: 1, Exceptions: 0
378
</pre>
379
            </p>
380
            <p>
381
                One of the main reasons for using a command line driven
382
                test suite is of using the tester as part of some automated
383
                process.
384
                To function properly in shell scripts the test script should
385
                return a non-zero exit code on failure.
386
                If a test suite fails the value <span class="new_code">false</span>
387
                is returned from the <span class="new_code">SimpleTest::run()</span>
388
                method.
389
                We can use that result to exit the script with the desired return
390
                code...
391
<pre>
392
&lt;?php
393
    require_once('simpletest/unit_tester.php');
394
    require_once('simpletest/reporter.php');
395
 
396
    $test = &amp;new GroupTest('File test');
397
    $test-&gt;addTestFile('tests/file_test.php');
398
    <strong>exit ($test-&gt;run(new TextReporter()) ? 0 : 1);</strong>
399
?&gt;
400
</pre>
401
                Of course we don't really want to create two test scripts,
402
                a command line one and a web browser one, for each test suite.
403
                The command line reporter includes a method to sniff out the
404
                run time environment...
405
<pre>
406
&lt;?php
407
    require_once('simpletest/unit_tester.php');
408
    require_once('simpletest/reporter.php');
409
 
410
    $test = &amp;new GroupTest('File test');
411
    $test-&gt;addTestFile('tests/file_test.php');
412
    <strong>if (TextReporter::inCli()) {</strong>
413
        exit ($test-&gt;run(new TextReporter()) ? 0 : 1);
414
    <strong>}</strong>
415
    $test-&gt;run(new HtmlReporter());
416
?&gt;
417
</pre>
418
                This is the form used within SimpleTest itself.
419
            </p>
420
 
421
        <p>
422
<a class="target" name="xml">
423
<h2>Remote testing</h2>
424
</a>
425
</p>
426
            <p>
427
                SimpleTest ships with an <span class="new_code">XmlReporter</span> class
428
                used for internal communication.
429
                When run the output looks like...
430
<pre class="shell">
431
&lt;?xml version="1.0"?&gt;
432
&lt;run&gt;
433
  &lt;group size="4"&gt;
434
    &lt;name&gt;Remote tests&lt;/name&gt;
435
    &lt;group size="4"&gt;
436
      &lt;name&gt;Visual test with 48 passes, 48 fails and 4 exceptions&lt;/name&gt;
437
      &lt;case&gt;
438
        &lt;name&gt;testofunittestcaseoutput&lt;/name&gt;
439
        &lt;test&gt;
440
          &lt;name&gt;testofresults&lt;/name&gt;
441
          &lt;pass&gt;This assertion passed&lt;/pass&gt;
442
          &lt;fail&gt;This assertion failed&lt;/fail&gt;
443
        &lt;/test&gt;
444
        &lt;test&gt;
445
          ...
446
        &lt;/test&gt;
447
      &lt;/case&gt;
448
    &lt;/group&gt;
449
  &lt;/group&gt;
450
&lt;/run&gt;
451
</pre>
452
                You can make use of this format with the parser
453
                supplied as part of SimpleTest itself.
454
                This is called <span class="new_code">SimpleTestXmlParser</span> and
455
                resides in <em>xml.php</em> within the SimpleTest package...
456
<pre>
457
&lt;?php
458
    require_once('simpletest/xml.php');
459
 
460
    ...
461
    $parser = &amp;new SimpleTestXmlParser(new HtmlReporter());
462
    $parser-&gt;parse($test_output);
463
?&gt;
464
</pre>
465
                The <span class="new_code">$test_output</span> should be the XML format
466
                from the XML reporter, and could come from say a command
467
                line run of a test case.
468
                The parser sends events to the reporter just like any
469
                other test run.
470
                There are some odd occasions where this is actually useful.
471
            </p>
472
            <p>
473
                A problem with large test suites is thet they can exhaust
474
                the default 8Mb memory limit on a PHP process.
475
                By having the test groups output in XML and run in
476
                separate processes, the output can be reparsed to
477
                aggregate the results into a much smaller footprint top level
478
                test.
479
            </p>
480
            <p>
481
                Because the XML output can come from anywhere, this opens
482
                up the possibility of aggregating test runs from remote
483
                servers.
484
                A test case already exists to do this within the SimpleTest
485
                framework, but it is currently experimental...
486
<pre>
487
&lt;?php
488
    <strong>require_once('../remote.php');</strong>
489
    require_once('../reporter.php');
490
 
491
    $test_url = ...;
492
    $dry_url = ...;
493
 
494
    $test = &amp;new GroupTest('Remote tests');
495
    $test-&gt;addTestCase(<strong>new RemoteTestCase($test_url, $dry_url)</strong>);
496
    $test-&gt;run(new HtmlReporter());
497
?&gt;
498
</pre>
499
                The <span class="new_code">RemoteTestCase</span> takes the actual location
500
                of the test runner, basically a web page in XML format.
501
                It also takes the URL of a reporter set to do a dry run.
502
                This is so that progress can be reported upward correctly.
503
                The <span class="new_code">RemoteTestCase</span> can be added to test suites
504
                just like any other group test.
505
            </p>
506
 
507
    </div>
508
<div class="copyright">
509
            Copyright<br>Marcus Baker, Jason Sweat, Perrick Penet 2004
510
        </div>
511
</body>
512
</html>