It was a bear, for sure (overvalued bugs)

 

Almost every year we go skiing in the mountains for about a week. We usually rent the same house  which is surrounded by a huge garden. One morning - it's quite a while back now  - we encountered a series of big footprints in the garden.  They were really large. Next to them was a big bloody bone which I assumed to be the mortal remains of a sheep. "What the heck happened here and whose footprints could these be?"

After measuring the size with a scale and analyzing the shape I put a 2-Swiss Frank coin into the track and muttered to myself.."this was clearly a bear".  I must add to this point, until this date, the only tracks I had ever reliably identified were those of a small rabbit. Regardless, everyone agreed, my wife, my kids and my parents in law who were joining us at holiday. It was enough confirmation to take some photos and show them to the local tourist information bureau. She too, was amazed by the photos. She picked the phone and called the forest ranger. While waiting at the counter and watching the officer talking with the ranger, I heard him mentioning a cat. Wait a minute....!

"Are you kidding me?", I protested. "Can you please tell the ranger about my photos and send those to him?", I added mortally offended. 

If at all one can mention a cat, then only because these footprints had the size of a mark from a cat's after-lunch nap in the snow. That's for sure.

The ranger promised to check our garden shortly before dawn and let us know. Of course, he never did and so I showed the photos to a local farmer. He confirmed, he had come to the same conclusion as me. He guessed, the ranger as well as the tourist office may both just have been afraid the story and the pictures could make it to the news and scare the tourists. 

That's good. Finally an expert who confirms what I've thought from the very beginning. Back home, two weeks later, I assorted my photos and decided to send a few of them directly to the ranger who never visited our holiday house. I was confident, once he sees the pictures, he'll be convinced and confirm my theory.

But, the ranger - who responded promptly - came up with a very interesting new theory. He claimed, it is not likely for the tracks being left by a bear. At the time the bears are usually still hibernating. He stated, these tracks were either left by a wolf or a big dog.

He explained, the huge size of the footprints may be caused by atmospheric conditions that let the footprints grow by at least 50% in size at relatively short time.

To be honest, I'd loved to hear something different , but it was an interesting argument and these statements made at lot more sense than the original comparison to a domestic cat. Of course, I am still not really convinced, but at least I am now at a point where I have to admit, there may be more than just my only one explanation.

 

 

 

How is that story related to testing? 

From time to time it happens that we find cool and unexpected bugs that have the potential of being a real big thing; the one killer bug. We might be euphoric and immediately record it and probably all to easy forget to collect more facts before we celebrate the great finding and label it highest priority. Often, when looking closer to an ugly looking bug and spending more time to understand its impact, the probability of the anomaly to occur in real life and the number of affected users, then what's left may be the sobering that we've just found another low, maybe medium priority bug, but we made a mountain out of a molehill upfront. It's great to find bugs, but be careful with the initial rating without having a second set of eyes looking at it.

By the way...., I am still convinced it was a bear, for sure =;O)

333-How much time do you need for testing?

According to Randall Rice [1], about one third of the total test effort should be spent on determining how to test. I’ve usually added another third to prepare reusable test data for regression tests and yet another third to finally execute the test. That’s what the 3-3-3 stands for me. This is for the current sprint where new features are being developed.

But I was often asked, how much time my team needs to perform a full manual regression test. While I could give a rough estimation, I liked to challenge the interviewer by asking whether she could provide me the number of bugs that I am going to find during our tests. I asked because I often experience tests that are scheduled for 30-45 minutes, taking up to 2 hours depending on the number of anomalies we find during the tests.

How can this happen?

• When an anomaly is detected, we need to analyze it. We need to understand whether it is a bug, a new feature or a function that we just don't remember working the way it is working now.
  
  
• We also need to prepare a reliable scenario to reproduce it and document it, otherwise product owners and developers tend to close the ticket too fast.
   
• In case the anomaly turns to be a defect, it is helpful to understand, when it slipped in. Understanding when the defect was introduced, impacts the priority of the issue product owners assign this finding. If the defect is there since weeks without anyone to notice, it may be considered with lower priority. If instead, it is a recently broken function, the priority set to this anomaly might be rated differently. Figuring out the exact date of intrusion can be time consuming. If you have automated tests, you can go through all past test results until you find the time when it still worked fine. Otherwise, you may need to run some extra tests on different test environments just to learn more about the history of the anomaly. This costs time, usually not planned in a manual regression test.
   
• What follows next is an accurate documentation for developers. One needs to create screenshots providing evidence of the feature how it worked before and how it behaves now that you found a problem. The more accurate the description the less time they lose understanding your ticket.
   
• Sometimes, an issue cannot easily be reproduced and you must examine the exact previous steps you took before the detection of the bug. Easy, if you followed a script; more challenging if you followed an exploratory approach while testing. The issue may occur in one particular situation of the workflow and may still work fine if you execute the workflow from a different starting point.
   
• If you are executing a test that someone else documented, you might find that important information is missing for you to effectively execute the test. You may need to ask someone for help or investigate your own, following referenced link (if they exist) and read additional material to finally understand what the purpose of the test was and how that feature-under-test is supposed to work.
   
• Testers are often asked for help when a problem occurs on production, because they often know more about the context and how and why a feature was built the way it is in production. The tester is taken away from the planned test to help someone in need. Such tasks cannot be planned.

These are all tasks that impact how far the estimation deviates from the actual time needed to perform a regression test. So, if you get asked how much time you need to test, my suggestion is to provide a rough estimate and add the disclaimer "...if we don't find any anomalies. Everything changes with every new bug we find".

T. Zelger, August 2020

 

References
[1] Testing Dirty Systems, by Randall W. Rice and William Perry

Empty Promises

...

Going Live with Gobla 1.0

Roast the test reports

Simplified Bug Prioritization Matrix

No cartoon today, it is just an illustration of our internal simplified bug prioritization matrix

Doozy! Only two bugs left.

That's what I thought, until we got the next version...

As James Whittaker wrote in “Exploratory Software Testing”, bugs tend to congregate for a variety of reasons such as complexity of the code, skill of the assigned developer, number of bugs in the past, etc. To make it short; where you see one bug there are likely more near around. You just need to look for them.

Bad luck !

According to the table below(published by the New York Times [1] today on March 20), New York has almost 7 times the amount of registered infected Corona patients compared to Washington but "only" about 0.5% of them have died so far.

That's quite a good quota (for Corona). Let's have a look at the other side of the continent.   The Corona deathrate in Washington is nearly 6%. That's quite a contrast. What can be the reason for this difference? In most other States the mortality rate is roughly around 2%, similar to other countries in the world.

Here are some superficial (not seriously meant) hypotheses:

• The New Yorkers are doing something extremly right and those in Washington are doing something extremely wrong

• The dirty air in New York is poison for the virus. The more CO2, the better for human beings. Bad news for Greta Thunberg.

• New York may not be a typical state for eldery people to relax...,probably more young and healthy people live there while in Washington may be a pan of sick or eldery people spend the rest of their lives there (again, this isn't meant seriously, okay?)

• New York was better prepared for this predictable disaster

• or...the measures may have been taken inconsistently and/or the numbers are simply wrong

Let's get more curious and read the full article. We find valuable information by reading the details and then very quickly understand what happened. It is noted that 2900 of the reported cases in New York were all registered at the day of publishing the numbers. Interesting, probably it is too early for those new cases to "die". If we ignore these new cases (7102-2900), we still have a quite good quote of 0.8%. Better than in many countries worldwide. But further reading the article reveals “At least 35 of the deaths were connected to a single nursing center in Washington" and "...many of those cases involved older people with other health problems that made them especially vulnerable to coronavirus”.

Aha! So, it was jsut bad luck for the nursing center to get hit so hard. If you ignore these numbers (75-35)…we are back to an almost "normal" rate of 2%.

These numbers make sense now. No need for me to look for creative hypothetical argumentation.

And the link to IT? A similar incident once happend near the end of our software development sprint. In parallel to the development of user-stories, we also performed refactoring tasks to get rid of technical debt. As is the nature of refactoring, it may break previously working functionality. This is what happened: Instead of being ready for the sprint review, we had to reject the demo of 2 buggy user-stories that were working properly just a day before. It cost us valuable story points. We didn't achieve the goal. What followed were long-winded discussions and assumptions of what to do and what not to do in the future, whereas I thought: "WTF, Just bad luck, shit happens".

[1]https://www.nytimes.com/interactive/2020/us/coronavirus-us-cases.html?action=click&module=Spotlight&pgtype=Homepage

Corona

Basel carnival cancelled, soccer games cancelled, business trips cancelled, Swiss Testing Day conference cancelled, borders closed and today it was announced that all flights from Europe to the States are going to be stopped by end of this week. WTF. My friends just managed to catch a plane to New Zealand...probably one of the last ones. Will they be able to catch a plane back home to manage their big farm in the Engadina? Everyday another employee is missing in our offices because he/she either feels sick or is afraid to feel sick soon. It's a weird silence. Those who dare to cough are potential suspects. Even worse if you fetch a handkerchief.

If you had predicated all this 2 weeks before, I had sent you to the psychiatrist to help you return to reality. Well, this has now become the new reality and that's why I feel baffled...where does all this end? Either, the whole racket goes up in smoke very soon or...yes, or what? Can't really think of any alternatives. It's a situation we never experienced before, hence whatever is decided probably can't be so wrong. It's all new, we are learning. We are learning to deal with a worst case scenario and we are likely to pay a high price. Can I go on a summer holiday and have a
swim in the Sea this year? Will we lose money if we book anything anywhere or do we have to stop planning at all and instead live spontaneously? Will I be trapped behind closed borders? Will my family fight for food soon? People living in war zones would still envy me for my comfortable situation. So it's not a worst case scenario yet, maybe just not a very funny one. What happens after this is all over? How many people will lose their jobs because their company went bankrupt or because the company is forced to tighten their belts. Spooky.


What shall I do? Drawing is a nice therapy...and while I am doing that, I am listening to Dave Grusin, Grover Washington, Fourplay, The Rippingstones, Earl Klugh, etc..how great this feels at the moment. It's my drug to disappear into a different world, at least for a short moment in time. I need it right now.

And tomorrow, I am having a beer with a good friend.

P.S.
Add on as of March 21: In general, this is a blog about software testing only where I put my cartoons and enrich it with explanatory text, but sometimes, I am inspired by things that happen outside the world of testing, like Corona for example. My brain is currently full of cartoon ideas related to Corona. Since I don't want to deviate from my testing related cartoon strategie, I have decided to simply put all my Corona related cartoons into this particular blog entry  rather than to create a new entry for each cartoon.

404 caused by clumsy whales

ThanX a lot to Balz Gilgen who handed over to me his draft for this cartoon.
I didn't have to change too much. I took it almost 1:1. Check yourself below.

Further reading on a real whale who got tangled:
https://www.newscientist.com/article/mg23231000-400-hacker-the-humpback-whale-who-got-tangled-in-an-internet-cable/

The Demo Effect

"Hang on, the demo starts soon, then let's go out and ruin their show".

Originally, the text was "wait until the sprint review is over, then let's go out and show up again", but the scene is less funny with the previous text. The current one turns this cartoon into a more common situation, aka. the demo-effect.

The original text  has its root in a real story. Long time ago (not in the company I work right now), the product owner regularly moved all reported defects to a low priority heap shortly before the sprint review, only to put them back into the next sprint right after the review. The goal was to shine with a good product having non-important bugs. With this approach he kept the release manager quiet, because the release manager was looking at high priority bugs only shortly before the review. If there weren't any left, the product owners were out of the line of fire.

It's like in Patriot Games, where the secret armed forces in the desert knew exactly when the spy satellite flied over their hidden military camp. They tided up everything shortly before it reaches their coordinates and then they rebuilt the camp after it had passed (until next time). Result: pin sharp satellite images of unsuspicious cabins in the desert.

Robots slacking off

Dear readers,
this is very likely my last cartoon for 2019 and I am already looking forward to drawing and publishing the next ones in 2020.
14 cartoons in a year is not bad, considering the fact, I have hardly any time to settle back after work and after having helped kids for school.

For those who want to know, I have a quite consistent hit-count on my blog (1600 pageviews p. month). In my world this is a lot. Well, not enough to become famous but still a lot of reasons to continue drawing in 2020.  I wish you very XMAS (even though I am kind of late) and a happy New Year 2020.
Torsten J. Zelger