Exploratory Test Method

Exploratory Test Method consists of the following steps:

1. Input Space Definition

2. Test Case Generation

3. Test Execution in Automatic Test Loop.

4. Verification of Outputs – Coverage Measurement and Pass/Fail Decision by Metamorphic Relation

5. Statistical Risk Analysis

Especially, some AI systems using training methods such as deep learning require more systematic verification because

they have a tendency to give unexpected results for unknown cases that they have ever met in their training.

Unfortunately, it is difficult to apply existing testing techniques to trained AI systems because most of testing techniques

use requirements specification that define relations of input to output precisely. Note that trained AI systems are trained

with data and have no specification for input-output relations.

Recently, the number of high risk AI systems such as autonomous robot and autonomous driving cars is increasing. Even

though these AI systems make new business opportunities by providing unmatched performance and functionalities,

highly self control nature of these systems may cause drastic risks.

Safety Engineering for AI