Beautifull Testing -Testing Open Source with Open Tools

This chapter in beautiful testing book reveals that testing anti-virus software can be considered as a challenging task.

Clam Anti Virus software project has developed a number of tools for testing their software. The methods of testing that they are using are Black box and White box testing.

Black box testing- A surface level testing which only considers the requirements specification.

White box testing- Internal implementation of the system is thoroughly considered and all the units of code is covered.

Statistics Analysis
1. A kind of white box testing
2. Look for common programming errors.
3. Ensure that the source code meets standards.
4. Manual and automatic testing are possible

GCC

1. Compilers are the mostly used tools for statistical analysis. And this is one of them.
2. Syntax and semantic(internal structure of the program) checking is done at the compilation time.
3. GCC refuses to compile code with bugs, therefore can identify and fix the bugs carefully.

Clang Static Analyser

1. Capable of finding critical buggs, such as devide by 0, invalid function calls etc.
2. When compared to other analysers this has a good advantage. That is; it provides a report in html format with the detected bugs.

Spling (Secure Programming Lint)

1. It can detect security-related issues such as null pointer dereferences,
memory management errors, possible infinite loops which other tools are not much capable of.
2. Splint is considered as a very sensitive tool.

Memory Checkers

1. Memory relate issues are captured in these types of testing tools.
2. Some of the examples for memory issues can be mentioned as memory leaks, using undeclared memory, writing out of bounds etc.

Valgrind

1. The advantage of this tool is that, we need not to modify the application that we need to test through it.
2. Can work with the binary files.
3. Can detect synchronization errors

Electric Fence and DUMA

1. Both are malloc() debuggers.
2. Can detect memory leaks
3. Can detect overruns of boundaries for heap-allocated buffers and invalid memory access.

Mudflap

1. It is a pointer debugging tool
2. It can detect memory reference violations
3. Mudflap is a part of GCC and additional components are therefore not required.
4. Can detect out of object access.

Unit Testing

1. This is a kind of white box testing
2. Used to check whether individual units of codes are working properly.
3. Unit tests allow us to quickly check the code before committing any changes to our SVN repository.
4. Can do unit testing in different platforms, this is one of the good advantages of a unit test
5. 'Check' is a popular open source unit-testing framework which is inspired by JUnit for C.

Test Scripts

1. A black box type of texting
2. It focuses only on the specification, not the internal details.
3. When once created, they can be executed automatically, making the testing faster.

Fuzz Testing

1.As the name implies

2.In fuzz testing we provide invalid, unexpected, or random data to the inputs of a computer program. 

3.Then the program is monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leaks. 

4.Fuzzing is mainly used to test for security problems in software or computer systems.

5.'Fuzil' is a tool which can be used for fuzzing.

Testing the Environment

1.'Autoconf' is a very popular tool which is used for testing the environment.

2. It configure script to test the system libraries and even the compilers for possible bugs

3. Some compilers create broken binaries while optimizing the code. These types of problems can be found in environment testing done by Autoconf type tools.