The challenge of prioritizing bugs

...

Duplicate Bugs Arguing in JIRA

I guess, we all agree, duplicate bug reports are a pain. They cause avoidable time to investigate. But, it is not always easy to find out whether a reported anomaly is already known.

 When a developer believes that a series of bug reports all have the same root cause, she tends to claim these bugs are all duplicates. The test engineer on the other hand would disagree and state “these are all different scenarios from an E2E perspective”.

 At the time of reporting an issue, we usually don’t know the root cause unless we dig deeper into understanding the anomaly. Even if a developer assures the bugs all have the same cause, it still makes no sense to mark these reports as redundant. One can never be sure the developer is right. I have experienced a lot of situations where a developer claimed having fixed a problem and then learnt that only part of the problem or a completely different one was repaired. 

 Michael Stahl [1] makes an interesting note when he states:

"Why would the same tester report the same issue twice? It just adds extra work for the tester, who for sure remembers the first report. Usually, a duplicate bug is reported when two testers identified the same problem and both reported it without first checking if it’s already in the system".

 I personally believe that doing an upfront research in the bug-tracking system doesn’t really help avoiding duplicates completely. We also contact our team mates or submit a question in a group chat.  A screenshot can help setting the record straight, but searching for clarifying pictures is even harder.

 When testers raise duplicate bug reports, we consider it an indication that these bugs are either easy to find and annoying.

 And customers? They don’t care about how many times the same bug was already reported. Some smart techies might "google" for a solution to their problems, but you can hardly avoid duplicate bug reports raised by customers. For example, according to Castelluccio [2], Mozilla receives hundreds of bug reports and feature requests from Firefox users every day. It is clear that in such cases, tools are required that categorize bug reports based on similarities of other bug reports to save a company’s time analyzing such anomalies.

 Per Runeson [3] describes an approach using NLP to support the automatic identification of duplicates. Their conclusion: "Even though only 40% of the duplicates are found using this approach, it still means a substantial saving for a major development organization"

 However, the best way to avoid duplicate bug reports is by fixing a reported anomaly straight ahead. Don’t wait for the duplicates.

---

References:

[1] When Testers Should Consider a Bug a Duplicate (Michael Stahl), Sticky Minds, January 15, 2018
[2] Teaching machines to triage Firefox bugs (Marco Castelluccio), April 2019 at https://hacks.mozilla.org
[3] Detection of Duplicate Defect Reports Using Natural Language Processing (Per Runeson & Magnus Alexandersson), IEEE 2007

Shortly before the Sprint Review

...

Aspiring killer bugs

My relationship to bugs is two-folded, actually it is a love-hate relationship. I hate 'em because they nest everywhere. There are places where you definitely don't want to have them, regardless whether you like to hunt bugs or not. Even an enthusiastic software tester doesn't like to see creatures in production turning customer's lifes into nightmare. But fact is, there is no space in software where there isn't enough potential for bugs hiding until they take their chance and show their ugly grimace. They are like cockroaches. Even if you think you have destroyed them all, there is always a few left.

But of course I also love these little virtual critters. Without bugs I had no job, or let's phrase is a little less dramatic; I'd probably had a different job and not such a great time putting them on paper.

Not all bugs cause headaches, but some actually do. Even the less dangerous ones can be a pain if they they show up in a pack. The job of a professional tester is to find as much of the critical ones as possible. That means, you have to study them; you need to understand where they "live" and and how they develop. Bugs aren't clever (at least the software bugs) but they use every tiny little whole you don't carburize.  Some wholes are so small that you can hardly see them until the bugs crawl out and make your program crash or behave in a very unexpected way. Often, you need to apply good techniques that make bugs sell themselves out because you wouldn't find them otherwise.

People with little to no dedication making software a great user-experience will not only miss important bugs, they also make a whole team look bad in front of a paying customer.
The job is by far not done if you simply compare deliverables from software developers with incomplete acceptance critieria defined in user-stories.

One great technique to find bugs is to apply test patterns when you investigate a feature. You probably cannot apply all test patterns to every feature, because some don't make sense depending on the context, but if you continously test with a checklist in mind, you will soon realize how much more there is to discover behind an innocent looking user-story.

Test patterns are described in many books. James Bach talks about testing heuristics, James Whittaker walks through a software like a tourist in a foreign country. The book "Lessons Learnt in Software Testing" by Cem Kaner, James Bach and Bret Pettichord is a great source of information to help you develop great testing ideas. If you take testing seriously, you have probably already developed your own set of test patterns, sprout from past mistakes, observations, experiences or lessons learnt from other testers and hopefully you have further developed those patterns, so your detection rate further increases. The list will never be completed but getting better and better.