Previous: When a test fails..., Up: Black-box tests

2.2.5 Why should I care?


Think of these tests as a mutual assistance pact: you should care about not breaking other people's code, because other people will care about not breaking your code.

Of course, this requires that you create black-box tests for your own code. Due to practicality, we can't check every single case of every single program. So instead, create one or two tests which investigate as many things as possible. For example, instead of simply testing if sfplay can output a sound file, we test changing the gain, starting at a specific time, and only playing for a specific length.

We recommend discussing potential black-box tests on the developer mailist.

New users

These tests are also very useful if you start investigating a new aspect of Marsyas. Currently there are many unmaintained MarSystems, applications, and projects in Marsyas. New users can easily waste hours trying to use part of the codebase which has been broken for months.

Having a testing mechanism means that users know that the code is working – at least, working for the exact command and input that the test uses. If (when?) a user has problems getting something to work in Marsyas, he can turn to the regtests: if a regtest passes but his own code doesn't work, he can compare his code against the regtest code.

Trust me, by now I've probably spent 20 hours trying to make broken code to work – sometimes when the developers already knew it was broken! Quite apart from the waste of time, it's incredibly demoralizing. It was this problem that inspired me to create these tests.