False Positives

When developers want to quickly demonstrate that in principle it works, they tend to hard-code things with the intention to replace that piece of code later. But, once it works, the focus is shifted to other tasks. It is likely that one forgets about these rotten code pieces.

 When Switzerland changed their tax rate from 7.6 per cent to 8 per cent, we were quite surprised that 7.6 was hard coded rather than read from a configuration file. As a result, we had to redeploy a new fix. In another example we found a code statement that made the program wait for 200 seconds. No wonder the application was slow. The developer had a good explanation why he added a wait-statement, but he mixed seconds with milliseconds.

 The term "false positives" has its roots in medical testing. A false positive is an error in which a test result improperly indicates the presence of a disease (the result is positive), when - in reality - it is not present, while a false negative is an error in which a test result improperly indicates no presence of the disease when in reality it is present.

Simply said, never trust any success messages. If the program claims it worked, the program may be wrong. Vice versa is also true. When the system throws an exception, the operation may still have worked correctly, regardless of the error.

 

 

Technical Debt not close-to-balance

I think it is normal in a project to accumulate technical debt somehow. You may have tough deadlines or other reasons to tend get things done more quickly than usual, probably with the mindset of getting it right later.

Other reasons for (quick) workarounds are new awareness (more experience) of how things work better under certain circumstances. The old approach was good for one particular problem, but not good enough for a general solution.

An increasing number of developers working on the same project may force the architect to enforce new coding guidelines or best practices for typical software development problems.

The use of embedded third party software may also trigger changes on your side when updates are delivered.

Technical debt should be avoided as much as possible and yes, there are scenarios where you are simply forced to live with technical debt. But, one should always be aware that, often, the time to improve old code won't be availble. Even if there will be such time, think about the risk of removing technical debt. Someone who works under pressure not only tends to seed ugly workarounds, but he is likely also adding sloppy unit tests (if he adds any at all). If that's the case, removing technical debts with refactored code adds new risks breaking functionality you won't know about until the customer reports them.

Kind of mixed up

Fred: "Lucy, look I bought a pedal tractor for our kids".

Lucy: "No you didn't, you fool! "

The challenge of prioritizing bugs

...

Commitment above-average

....

How dare you, Lambretta?

Usually I am testing software, but today, I enjoyed a 15 minute test ride on a brand-new Lambretta and
I decided to add my personal results to my cartoon blog even though this blog is mainly for software testing.

My goal today was to check-out whether the Lambretta revival V-Special is an interesting alternative
to my beloved but seriously broken Vespa. I've read the Lambretta advertisement in the news. I checked many pictures and it made me curious. The longer I googled the more excited I got, and I also checked the price which seemed reasonable to me.

The positive things I noted, was the quiet motor and the firm road holding at the speed of 80km/h. Something I wasn't used to my current Vespa. Also the design of the new Lambretta is great and contains so much of the Sixties. There is also space for the helmet under the seat which is cool.
However, less joyful is the fact, that almost everything is made out of plastic. Wouldn't it be nice if at least the wings were solid metal. If you carry a passenger, the wings will be full of scratches shortly.
In order to save fuel, you need to consider weight, but a little less of elastic material is desirable.

While on the test ride, I realized the slightly oversized LCD monitor just below the small analog speedo. Unfortunately, the LCD is so big that you can hardly read your speed in the analog display which I think is kind of dangerous, because you spend too much time looking for the needle; time that may be needed elsewhere.

Overall it looks really nice. The designers did a great job keeping as much of the Sixties as possible.The many plastic is a disappointement. I really hope they are going to change that soon.

Friday afternoon at Green Galactica

...

He just knew too much

THE DAILY BUG. "A software tester was found dead due to an overdose of drugs yesterday evening at his home in lower Manhattan, New York. His body was discovered before dawn on Sunday, October 5. He was a professional software tester and known for his strategies in spotting even hard to find bugs. An interviewed friend on site believes, he was murdered. "The tester was threatened several times by suspicious phone calls", he said. The police states they have found traces left by ugly critters at the crime scene. The police also stated, the main suspect was identified as Dendro Ponderosa, a modified name which has its roots in the Latin name Dendroctonus ponderosae, a mountain pine beetle and a remote relative of Dendro. The police published an appropriate mug shot of the suspect."
Reported by Formica Rufa

BETA Test at Green Galactica

One day the PL asked me to provide a checklist with the most important acceptance criteria and which served as checkpoints to enter the next phase of the project. The list was later extended with a few more items by the PL and PO. When the date came closer, someone worked on the list and changed most of the quality attributes from MUST to NICE-TO-HAVE without upfront warning. Obviously, someone realized one is not going to meet all the goals and changed the attributes beforehand. As a quality manager, I was concerned that tasks labelled NICE-TO-HAVE are never picked up, therefore I protested heavily.

 We then agreed to keep the list with its original priority, and, on D-DAY we would go through the list again and then decide for each unfinished task how bad it really is and what are the risks related if we are not completing the task.

 But, the cartoon fits even better to several similar situations where the testers warned management long in advance about the immaturity of the system. When deadlines were close, the pattern was often the same: concerns were overruled, management had “higher” goals and caused a mess with the decision to rollout the software despite the obvious consequences.

Hiding bugs

Duplicate Bugs Arguing in JIRA

I guess, we all agree, duplicate bug reports are a pain. They cause avoidable time to investigate. But, it is not always easy to find out whether a reported anomaly is already known.

 When a developer believes that a series of bug reports all have the same root cause, she tends to claim these bugs are all duplicates. The test engineer on the other hand would disagree and state “these are all different scenarios from an E2E perspective”.

 At the time of reporting an issue, we usually don’t know the root cause unless we dig deeper into understanding the anomaly. Even if a developer assures the bugs all have the same cause, it still makes no sense to mark these reports as redundant. One can never be sure the developer is right. I have experienced a lot of situations where a developer claimed having fixed a problem and then learnt that only part of the problem or a completely different one was repaired. 

 Michael Stahl [1] makes an interesting note when he states:

"Why would the same tester report the same issue twice? It just adds extra work for the tester, who for sure remembers the first report. Usually, a duplicate bug is reported when two testers identified the same problem and both reported it without first checking if it’s already in the system".

 I personally believe that doing an upfront research in the bug-tracking system doesn’t really help avoiding duplicates completely. We also contact our team mates or submit a question in a group chat.  A screenshot can help setting the record straight, but searching for clarifying pictures is even harder.

 When testers raise duplicate bug reports, we consider it an indication that these bugs are either easy to find and annoying.

 And customers? They don’t care about how many times the same bug was already reported. Some smart techies might "google" for a solution to their problems, but you can hardly avoid duplicate bug reports raised by customers. For example, according to Castelluccio [2], Mozilla receives hundreds of bug reports and feature requests from Firefox users every day. It is clear that in such cases, tools are required that categorize bug reports based on similarities of other bug reports to save a company’s time analyzing such anomalies.

 Per Runeson [3] describes an approach using NLP to support the automatic identification of duplicates. Their conclusion: "Even though only 40% of the duplicates are found using this approach, it still means a substantial saving for a major development organization"

 However, the best way to avoid duplicate bug reports is by fixing a reported anomaly straight ahead. Don’t wait for the duplicates.

---

References:

[1] When Testers Should Consider a Bug a Duplicate (Michael Stahl), Sticky Minds, January 15, 2018
[2] Teaching machines to triage Firefox bugs (Marco Castelluccio), April 2019 at https://hacks.mozilla.org
[3] Detection of Duplicate Defect Reports Using Natural Language Processing (Per Runeson & Magnus Alexandersson), IEEE 2007