The challenge of prioritizing bugs

...

Commitment above-average

....

How dare you, Lambretta?

Usually I am testing software, but today, I enjoyed a 15 minute test ride on a brand-new Lambretta and
I decided to add my personal results to my cartoon blog even though this blog is mainly for software testing.

My goal today was to check-out whether the Lambretta revival V-Special is an interesting alternative
to my beloved but seriously broken Vespa. I've read the Lambretta advertisement in the news. I checked many pictures and it made me curious. The longer I googled the more excited I got, and I also checked the price which seemed reasonable to me.

The positive things I noted, was the quiet motor and the firm road holding at the speed of 80km/h. Something I wasn't used to my current Vespa. Also the design of the new Lambretta is great and contains so much of the Sixties. There is also space for the helmet under the seat which is cool.
However, less joyful is the fact, that almost everything is made out of plastic. Wouldn't it be nice if at least the wings were solid metal. If you carry a passenger, the wings will be full of scratches shortly.
In order to save fuel, you need to consider weight, but a little less of elastic material is desirable.

While on the test ride, I realized the slightly oversized LCD monitor just below the small analog speedo. Unfortunately, the LCD is so big that you can hardly read your speed in the analog display which I think is kind of dangerous, because you spend too much time looking for the needle; time that may be needed elsewhere.

Overall it looks really nice. The designers did a great job keeping as much of the Sixties as possible.The many plastic is a disappointement. I really hope they are going to change that soon.

Friday afternoon at Green Galactica

...

He just knew too much

THE DAILY BUG. "A software tester was found dead due to an overdose of drugs yesterday evening at his home in lower Manhattan, New York. His body was discovered before dawn on Sunday, October 5. He was a professional software tester and known for his strategies in spotting even hard to find bugs. An interviewed friend on site believes, he was murdered. "The tester was threatened several times by suspicious phone calls", he said. The police states they have found traces left by ugly critters at the crime scene. The police also stated, the main suspect was identified as Dendro Ponderosa, a modified name which has its roots in the Latin name Dendroctonus ponderosae, a mountain pine beetle and a remote relative of Dendro. The police published an appropriate mug shot of the suspect."
Reported by Formica Rufa

BETA Test at Green Galactica

One day the PL asked me to provide a checklist with the most important acceptance criteria and which served as checkpoints to enter the next phase of the project. The list was later extended with a few more items by the PL and PO. When the date came closer, someone worked on the list and changed most of the quality attributes from MUST to NICE-TO-HAVE without upfront warning. Obviously, someone realized one is not going to meet all the goals and changed the attributes beforehand. As a quality manager, I was concerned that tasks labelled NICE-TO-HAVE are never picked up, therefore I protested heavily.

 We then agreed to keep the list with its original priority, and, on D-DAY we would go through the list again and then decide for each unfinished task how bad it really is and what are the risks related if we are not completing the task.

 But, the cartoon fits even better to several similar situations where the testers warned management long in advance about the immaturity of the system. When deadlines were close, the pattern was often the same: concerns were overruled, management had “higher” goals and caused a mess with the decision to rollout the software despite the obvious consequences.

Hiding bugs

Duplicate Bugs Arguing in JIRA

I guess, we all agree, duplicate bug reports are a pain. They cause avoidable time to investigate. But, it is not always easy to find out whether a reported anomaly is already known.

 When a developer believes that a series of bug reports all have the same root cause, she tends to claim these bugs are all duplicates. The test engineer on the other hand would disagree and state “these are all different scenarios from an E2E perspective”.

 At the time of reporting an issue, we usually don’t know the root cause unless we dig deeper into understanding the anomaly. Even if a developer assures the bugs all have the same cause, it still makes no sense to mark these reports as redundant. One can never be sure the developer is right. I have experienced a lot of situations where a developer claimed having fixed a problem and then learnt that only part of the problem or a completely different one was repaired. 

 Michael Stahl [1] makes an interesting note when he states:

"Why would the same tester report the same issue twice? It just adds extra work for the tester, who for sure remembers the first report. Usually, a duplicate bug is reported when two testers identified the same problem and both reported it without first checking if it’s already in the system".

 I personally believe that doing an upfront research in the bug-tracking system doesn’t really help avoiding duplicates completely. We also contact our team mates or submit a question in a group chat.  A screenshot can help setting the record straight, but searching for clarifying pictures is even harder.

 When testers raise duplicate bug reports, we consider it an indication that these bugs are either easy to find and annoying.

 And customers? They don’t care about how many times the same bug was already reported. Some smart techies might "google" for a solution to their problems, but you can hardly avoid duplicate bug reports raised by customers. For example, according to Castelluccio [2], Mozilla receives hundreds of bug reports and feature requests from Firefox users every day. It is clear that in such cases, tools are required that categorize bug reports based on similarities of other bug reports to save a company’s time analyzing such anomalies.

 Per Runeson [3] describes an approach using NLP to support the automatic identification of duplicates. Their conclusion: "Even though only 40% of the duplicates are found using this approach, it still means a substantial saving for a major development organization"

 However, the best way to avoid duplicate bug reports is by fixing a reported anomaly straight ahead. Don’t wait for the duplicates.

---

References:

[1] When Testers Should Consider a Bug a Duplicate (Michael Stahl), Sticky Minds, January 15, 2018
[2] Teaching machines to triage Firefox bugs (Marco Castelluccio), April 2019 at https://hacks.mozilla.org
[3] Detection of Duplicate Defect Reports Using Natural Language Processing (Per Runeson & Magnus Alexandersson), IEEE 2007

Recursive Programming transferred to Drawings

Shortly before the Sprint Review

...

Persistent Positive Thinking

...

Confetti User Stories

Finding the right balance between "too big" and "too small" isn't easy sometimes.

Lining up in the Ocean

It's about how fast we can be wrong with our conclusions. I often see that we detect an alledged new issue, not realizing that the same anomaly was already there before. It's easier and faster to judge without collecting the facts first. Things that may look the same, sometimes aren't.

Patched UFOs

When developers state, they didn't change anything but one line of code, they actually believe that themselves. It's just that they have a bias towards forgetting all the work they've done (plus their colleagues') before their last coffee break.

Sprint Review #37

An agile team is one that continually focuses on doing its best work and delivering the best possible product. In our experience, this involves a ton of discipline, learning, time, experiementation, and working together.
[Crispin - Agile Testing]

Gung-Ho Tester

Last Friday, in the "daily", Cindy handed over a draft of her own doodled sketch about me. I was really surprised and found it brilliant. I promised her to put the sketch into my style cartoon over the weekend and here we go.

ThanX. I never met someone who made a drawing about me. I really enjoyed having this moment of conducting a little self-mockery.









And this is the original sketch from Cindy:

Analyzing Lenny's flight paths

We are tracking the routes of animals all over the world. Some birds have already become famous like Lenny the stork whose annual air trip to Spain and back to his home at the Basel Zoo is observed by over 1500 fans. Lenny has a transmitter and everyone can check online when he starts his journey from Switzerland to South Spain where he spends a few months before he returns.

His routes [lenny] are recorded since 2013. Each time he starts the way back to his hometown, the local news celebrate it with corresponding articles [lenny2]. Lots of excited people are await his arrival near February/March. They are fascinated by the fact he returns to the same nest and same "girlfriend" every year to enlarge his family. What a faithful and diligent bro.

I wonder what Lenny thought of us bird-stalkers if he knew that each day of his life is tracked as a spot on a map that many people find so exciting. He'd probably bring us to court, took off the transmitter or at least got himself unsubscribed from all social media accounts. 

Maybe he'd become more vain and demonstrate that he could fly further south to Africa like others and/or his ancestors did. He may probably also felt ashamed when he realized we are watching him stop at all these fast-food rest places that are nothing else but garbage dumps. Or, he could be a little rascal like in the cartoon - with a big portion of humour. But since he doesn't know he's getting observed, he blithely goes on doing what many others do until a yet unknown trigger in his head tells him to go back home.

It's is not only the incredible flight-paths that I find so fascinating, it is also sad and heartrending if you see that almost every second of the ringed storks do not survive. The data logger web page [logger] keeps a record of each stork with his/her name and if you read their fates, you will learn that some don't even manage to accomplish a single trip. They die by flying into electric power lines, wind wheels, get shot or perish because they ate wrong or morbid things. These are all causes of death which are not natural.

But that's not really what I was about to talk. While digging deeper into Lenny's flight routes, I got a lot of questions. Why do these storks no longer fly to Africa and instead stay in Spain? How do they know where to go and how do they find their way back to their original nest they grew up? When do they start going South and how do they know it's time to return?

Scientists have raised the same questions about the truncated routes and they have come up with two theories:

1. The Swiss storks are the result of a resettlement project that started with Algerian storks, a species that is not used to fly the same long distances as the original domestic storks. The route they are taking now pretty much matches the distance to what their relatives are "programmed for" or what they can perform.
   
   
2. The second theory is less pleasant. Spain is full of garbage dumps where they find enough food. If you follow the markers of Lenny and then switch from the "map"-view to "satellite"-view, it is really appalling to see where he likes to rest and eat. I feel sorry for him and his friends. These garbage dumps seem to be another reason why there is no need to search for food further south.

These are fair explanations, but I could not find any answer to all my other questions like how do they remember which route to take and when? Explanations in scientific articles range from "inner-clock", "genetically programmed to go South with no plan" to "orientation by the Sun", "stars" and even "magnetic fields" or "orientation by the landscape".

I am a software tester. If I am given similar amounts of illustrations from a software developer whose components don't work, then my conclusion is: he simply doesn't know.

I invested some time and analyzed just one of those theories, "orientation by the landscape".

If you zoom out then it looks like Lenny takes the exact same route every year. But if you zoom into Lenny's flight path, I see that his first 2 routes to Spain went over the region of Burgundy in France while the last 2 years he took a route which was 100 km East of the first routes. This doesn't really look to me as if he strongly remembers each hill and course of a river. Anyhow, it still looks like he follows a main path from which he takes small excursions.

When Lenny headed South the very first time, he took a shortcut over the Mediterranean Sea (08/2013); a route which he never took again in the subsequent years on his way South. But he used it regularly on his way back north except for 2016 and 2017. That's interesting, because cause I was told, big birds don't like to fly over the sea. They have less natural draft. It costs them more energy to fly. So, what happened the first time that he never took that route again? And why did he find that route more attractive for his return to Switzerland? Did he follow and  trust less experienced storks?  Probably not. When Lenny was chipped in 2013, he was already an experienced 10 year old stork. May weather conditions or seasonable winds play a role? Can these parameters influence the conditions and allow for a more or less economical flight over the Sea at some time of the year? Did he learn about dangerous places which he tried to avoid next time? Is this aberration from the main route broadening his skills and make him a real travel expert over time by learning on the fly and really remember all these things?

Now we have our link to software testing. 

Isn't there an analogy to the process of following a strict test script vs. taking some detours? What if we don't enter the prescribed number 9481 in the calculator but do something different and watch what happens next? 

What if we use an alternative way to trigger the same software function/operation? I can enter a text by typing or by pasting from the clipboard into an edit field and I swear to you it is not the same! Isn't the habit of taking detours rewarding us with th e detection of more interesting bugs?

If you test software for many years, you will get a better sense of where things can go wrong. No book can teach you that. There are interesting books and articles that go into this direction like those from James Whittaker [whit], but it's not the same like experiencing some pain on your own. 

 

What about subject matter experts? They are like the elder storks. You would be dumb not to take their advice. They may not be the best testers, but they might lead you to very interesting places you had never been before.

Sources:
[lenny]
http://storch-schweiz.danielbischof.de/map/2964

[lenny2]
https://www.srf.ch/news/regional/basel-baselland/heimkehr-aus-dem-sueden-storch-lenny-ist-im-anflug 

[logger]
https://projekt-storchenzug.com/datenlogger/

[whit] 
Exploratory Software Testing by James Whittaker