Here you can find the raw data used in our statistical analysis.
- Data used for evaluating the effectiveness (fault detection loss) and efficiency (time) of the proposed approach in the inadequate (budget) scenario:
- Data used for evaluating the effectiveness (test suite reduction/fault detection loss) and efficiency (time) of the proposed approach in the adequate scenario:
- Data used for evaluating the approach in the large-scale scenario:
Box plots for Fault Detection Loss in the inadequate (budget) scenario:
Fault Detection Loss in the inadequate (budget) scenario:
Reduction times for the budget scenario (including and excluding preparation time):
Box plots for Test Suite Reduction and Fault Detection Loss in the adequate scenario:
Test Suite Reduction and Fault Detection in the adequate scenario:
Reduction times for the adequate scenario (including and excluding preparation time):
Time required to reduce 500k test cases to different reduction targets:
Time and space needed to compute and store prepared data by FAST-R in the large-scale scenario:
We have also applied the FAST-R family to a synthetic set of coverage data for the large-scale test suite, and could also assess the scalability of the adequate versions of FAST-R. The results are displayed in the table below (each value is the average after 50 repetitions).
| Approach | Test Suite Reduction Achieved | Preparation Time | Selection Time |
|---|---|---|---|
| FAST++ | 99.99% | 314.43 | 194.38 |
| FAST-CS | 99.99% | 314.43 | 39.62 |
| FAST-all | 47.89% | 12979.29 | 61.68 |
| FAST-pw | 99.99% | 12979.29 | 281.81 |