Monday, October 25, 2010

Revolt of the Test Scripts

If testers are not invited early enough, they will not have the chance and the time to determine whether or not they have to adapt their existing scripts. As a matter of fact, some scripts will have to be de-activated since they start to report errors that aren't any. Someone smart once called this "False negatives", consequently the opposite exists, too (false positives). see also the blog entry as of July 2009 (The Test Successfully Failed). the way, Test Tools and/or your own test scripts can be buggy, too (here a false negative). the meantime, I found out under which conditions this situation occurs. In the NUnit Test Automation Framework, even if all your test scripts were running fine the whole suite may report a failure if someting goes wrong outside the test suite. The TearDown function is such a method that gets called automatically after each script and/or after the suite is completed. At that point, all scripts reported already a successful run, but the TearDown can still throw an exception if for instance your implementation invokes a call of a null object (null pointeer exception). In such case, NUnit reports a failed suite although all child scripts were passing all tests.


  1. Maybe the test scripts are revolting because they can never be used usefully to test :)

  2. Oh, they can...but they're like little children. If you don't take care of them, they will do what they want...=;O)

  3. Love the cartoon. Would like to contact you about permission to use in a slide for a public talk (with attribution of course).

    1. Hi testmuse
      No problem, you can use it. Go ahead and thanX for mentioning the site
      Best regards