An experimental study on test suite characteristics, cost, and effectiveness of FSM-based testing methods.
Título da Revista
ISSN da Revista
Título de Volume
Testing from finite state machines has been widely investigated due to its well-founded and sound theory as well as its practical application in different areas, e.g., Web-based systems and protocol testing. There has been a recurrent interest in developing methods capable of generating test suites that detect all faults in a given fault domain. However, the proposal of new methods motivates the comparison with traditional methods. In this context, we conducted a set of experiments that compares W, HSI, H, SPY, and P methods. Different parameters of the test suites were compared, such as test suite characteristics, test suite length, and fault detection ratio. These dimensions were analyzed with different machine configurations, varying number of states, inputs, outputs, and transitions. The results have shown that H, SPY, and P methods produce significantly smaller test suites than traditional methods (W, HSI). The P, H and SPY methods presented test suites of similar length, though P produced test suites which are sligthly shorter. We have also observed that the reduction ratio in partial machines is smaller than that in complete machines. On average, the recent methods produce test cases with lengths longer than traditional methods. SPY and P methods have the highest fault detection ratios and the HSI method presents the lowest ratio. Several correlations were tested and identified among the different configurations evaluated.