Software Testing - Mutation Testing

Mutation Testing is a white-box testing technique used to measure how effective your test cases are.
It works by intentionally introducing small changes (mutations) into the program's source code to check whether the existing test cases can detect these changes.

If your tests fail when the code is mutated, the mutation is killed.
If your tests still pass, the mutation survives — meaning your test suite is weak.

What Is Mutation Testing?

In Mutation Testing, the system:

  1. Takes the original code

  2. Makes tiny modifications (mutants)

  3. Runs the existing test cases

  4. Checks whether tests catch the mutation

The goal is to evaluate test case strength, not application correctness.

Why Is It Called Mutation Testing?

Because the testing tool creates multiple mutated versions of the code, similar to “mutations” in biology.
Each mutated version is tested to verify if the test cases can detect the changes.

Purpose of Mutation Testing

  • To check quality and coverage of test cases

  • To identify weak or missing test cases

  • To ensure code is thoroughly tested

  • To improve reliability of unit test suites

How Mutation Testing Works (Step-by-Step)

  1. Original Code is taken as base

  2. Mutations are introduced

    • Usually small changes like operators, variables, or conditions

  3. Test Suite is executed on each mutant

  4. Results are observed

    • If tests fail → mutation killed

    • If tests pass → mutation survived

  5. Improve test cases to kill surviving mutants

Common Types of Mutations

1. Operator Replacement

Changing arithmetic operators

  • + → -

  • * → /

2. Logical Operator Changes

  • && → ||

  • != → ==

3. Variable Replacement

Replacing a variable with another

4. Constant Modification

  • 5 → 0

  • true → false

5. Statement Removal

Removing a line of code or condition

These mutations help reveal gaps in the test suite.

Example of Mutation Testing

Original Code

if (age >= 18) {
    return "Eligible";
} else {
    return "Not Eligible";
}

Mutant 1

>= changed to >

if (age > 18) {

Mutant 2

Return values swapped

return "Not Eligible";

Mutant 3

Condition removed

return "Eligible";

Test Case Result Example

  • If tests detect the wrong output → mutant killed

  • If tests do not detect the change → mutant survived

A surviving mutant means the test suite is not covering boundary or logic cases properly.

Mutation Score

To measure test quality:

Mutation Score = (Killed Mutants / Total Mutants) × 100

Higher score = Stronger test suite

Mutation Testing Tools

  • PIT Mutation Testing (Java)

  • MutPy (Python)

  • Javalanche

  • Stryker (JavaScript, TypeScript)

  • MuJava

  • Ninja Mutant

Benefits of Mutation Testing

  • Improves test case effectiveness

  • Helps uncover missing test scenarios

  • Ensures high code coverage

  • Enhances software quality and reliability

  • Detects weak conditions, edge cases, and logical flaws

Limitations

  • Computationally expensive (many mutants created)

  • Time-consuming for large projects

  • Not suitable for UI or integration tests

  • Mostly used for unit testing, not system testing