2.2.1 What does "black-box" mean?

Definition

The term “black-box testing” seems to mean something different to each project, website, and software engineering academic. So I shall define what we mean by “black-box testing” in the context of Marsyas “black-box testing”:

Does it work the way it used to?

This is checked as follows:

1. Pick a program to test. Find some input for the program (generally a sound file). Save the input.
2. Run the program on the saved input. Save the output (either another sound file, or some text).
3. Wait / change stuff / let other people change stuff.
4. Run the program with the initial input. Compare the current output to the initially stored output. If they are different, the test fails.
5. Goto step 3.

In other words, these tests do not check for correctness; they simply check for consistency. Fixing a bug could result in a test “failing”. However, this is not a problem: see When a test fails...