What Is a Good Test Case

[attachment=16977]
What’s a Test Case?
Let’s start with the basics. What’s a test case?
IEEE Standard 610 (1990) defines test case as follows:
“(1) A set of test inputs, execution conditions, and expected results developed for a
particular objective, such as to exercise a particular program path or to verify compliance
with a specific requirement.
“(2) (IEEE Std 829-1983) Documentation specifying inputs, predicted results, and a set
of execution conditions for a test item.”
According to Ron Patton (2001, p. 65),
“Test cases are the specific inputs that you’ll try and the procedures that you’ll follow
when you test the software.”
Boris Beizer (1995, p. 3) defines a test as
“A sequence of one or more subtests executed as a sequence because the outcome and/or
final state of one subtest is the input and/or initial state of the next. The word ‘test’ is
used to include subtests, tests proper, and test suites.


Information Objectives
So what are we trying to learn or achieve when we run tests? Here are some examples:

Find defects. This is the classic objective of testing. A test is run in order to trigger
failures that expose defects. Generally, we look for defects in all interesting parts of the
product.

Maximize bug count. The distinction between this and “find defects” is that total number
of bugs is more important than coverage. We might focus narrowly, on only a few highrisk
features, if this is the way to find the most bugs in the time available.

Block premature product releases. This tester stops premature shipment by finding bugs
so serious that no one would ship the product until they are fixed. For every releasedecision
meeting, the tester’s goal is to have new showstopper bugs.

Help managers make ship / no-ship decisions. Managers are typically concerned with
risk in the field. They want to know about coverage (maybe not the simplistic code
coverage statistics, but some indicators of how much of the product has been addressed
and how much is left), and how important the known problems are. Problems that appear
significant on paper but will not lead to customer dissatisfaction are probably not relevant
to the ship decision.

Minimize technical support costs. Working in conjunction with a technical support or
help desk group, the test team identifies the issues that lead to calls for support. These are
often peripherally related to the product under test--for example, getting the product to
work with a specific printer or to import data successfully from a third party database
might prevent more calls than a low-frequency, data-corrupting crash.

Assess conformance to specification. Any claim made in the specification is checked.
Program characteristics not addressed in the specification are not (as part of this
objective) checked.

Conform to regulations. If a regulation specifies a certain type of coverage (such as, at
least one test for every claim made about the product), the test group creates the
appropriate tests. If the regulation specifies a style for the specifications or other
documentation, the test group probably checks the style. In general, the test group is
focusing on anything covered by regulation and (in the context of this objective) nothing
that is not covered by regulation.

Minimize safety-related lawsuit risk. Any error that could lead to an accident or injury is
of primary interest. Errors that lead to loss of time or data or corrupt data, but that don’t
carry a risk of injury or damage to physical things are out of scope.

Find safe scenarios for use of the product (find ways to get it to work, in spite of the
bugs). Sometimes, all that you’re looking for is one way to do a task that will consistently
work--one set of instructions that someone else can follow that will reliably deliver the
benefit they are supposed to lead to. In this case, the tester is not looking for bugs. He is
trying out, empirically refining and documenting, a way to do a task.


Function Testing
Test each function / feature / variable in isolation.
Most test groups start with fairly simple function testing but then switch to a different style, often
involving the interaction of several functions, once the program passes the mainstream function
tests.


Risk-Based Testing
Imagine a way the program could fail and then design one or more tests to check whether the
program will actually fail that in way.
A “complete” set of risk-based tests would be based on an exhaustive risk list, a list of every way
the program could fail.