Beautiful Testing - Beautiful Mutation Testing - chapter 18

In mutation testing what we are doing is; using a slightly and systematically changed version of the program which has defects, and test against our test suit to find whether our test suite can find them.

For an example consider the bellow example code.

if (x||y) {
  z = 1;
}
else{
  z = 0;
}

We can mutate the above code by changing the logical operator as bellows.

if (x&&y) {
  z = 1;
}
else{
  z = 0;
}

Evaluating test case quality

There are several metrices for evaluating test case quality.

1. Code coverage
All the codes are needed to be executed by the test suit without bypassing any.
2. Condition coverage
All the condition statements are needed executed by the test suit

The AspectJ compiler is an extention to java that can be used for testing.
In this chapter it is used Aspectj to view the bug distribution along different classes of a project. It is then obvious that the bud distribution is uneven.
Pareto effect : When the distribution of bugs are uneven then it is called the pareto effect.

According  to the book mostly the system tests are practically carried out through Junit which is used for unit testing in java.

The book has used the term "watching the watchmen" metaphorically to figure out that the 'quality of quality assurance' should be tested.

Mutations : These are the artificial defects introduced to a program. ex: such as changing a constant.
If the test suite is able to find it then, we can be sure that it could find the real bugs.

Benefits of Mutation Testing

1. Can evaluate the test suite's capability
2. Can identify modules with high risk

Some mutation frameworks available for every one;

1. Mothra - For Fortrun
2. uJava - For Java

Drawbacks of Mutation Testing

1. Very time consuming

Tips to make mutation testing efficient;

1. Mutate the binary code so that rebuilding is avoided
2. Use mutant scemata
3. Ignore uncovered code

Adding a mutation;

1. Replace numerical constant
2. Negate jump conditions
3. Replace arithmetic operations
4. Removing method calls

Equivalent mutations: Some operators has the capability to create a similar program like the previous program even after adding a mutation. This results in an equivalent mutation.

Javalanche Framework

1. This is used for mutation testing of Java programs.
2. It is a free for everyone framework.
3. And the other thing is that, it can be extended with additional operators or impact detectors.