Would Heu-risk it? Part 20: Drowning in the deep

picture of a diver, looking at a small squid. Below him/her, a gigantic tentacle monster is reaching upwards

Ok ok, I know I warned you about the shallows before. And now this trap warns of the opposite! Can I have it both ways?! Eat the cake and keep it? Well, let us check out the rhyme and then see if I can sort it out for us:

”I´m glad you dared to take that leap
Hunt the treasures hidden deep
But beware, just don´t get caught
And end up drowning, learning naught”

So, what does it mean?

What could possibly be wrong with testing ”too deep”? Being ”too thorough”? We just want to do our job well, right?
Well. If you happen to be in the situation where you can spend any time you like on every story, good for you!
Most of us can’t.
We always (I know. I said never to use always) have a context, constraints, risks and goals to take into considerations.

Also, it is a complete waste of money and some problems are better found by skimming the surface. A balance, as usual, is best.
If the gain of releasing now is higher than the gain of finding a few more bugs – we should stop.

If the cost of continuing testing is higher than the probability of finding more if we dig deeper – we should stop.
If we have safety nets in place and confidence that we can fix problems quickly if they are found in production – why should stop.
All of those, and any other we can come up with, are just beliefs of course. There is no way of knowing that the bugs still existing within the code are not of the kind that will destroy everything. We need to think, assess risk and take decisions based on what we know.

There is also the risk that spending all that time testing one thing means we will not have time to properly test something else. And that can be very bad.
If we always strive to be 100% confident in everything, we will never release anything!

”If I waited for perfection, I would never write a word”

Margaret Atwood

But. A lot of testers are a bit on the control-freaky side. And we like to understand things. Take things apart until we know what makes it tick.
And it can be so incredibly tempting to just try one more thing. We almost have it! We can taste the bug in the air but it keeps eluding us.

Bildresultat för tester meme

And to be honest – after a while, we start getting blind spots. If you have been chasing that ”maybe problem” for too long, you start running in circles. It might be easier to find that problem by actually stopping, taking a break and test something else. If there is indeed a problem, I can almost guarantee it will come back to you at a later point in time.

Story time

Not that I don’t have any stories of me getting stuck in this hole but today I am choosing a story from one of my ”Speedtesting with mind maps”-workshop with Tomas Rosenqvist.

We were doing the workshop with a group of students and we nudged a few of the pairs to look a bit deeper into a particular problem. We wanted to get them to not get distracted with the shallow issues but try to use their skills to figure out why a certain scenario seemed to be off.

Unfortunately, our nudge instead got them totally fixated with that problem and no matter what they tried they could not figure it out. Even though we hinted a few times that time was passing, they had completely committed to that bug and ended up not having time to test the other parts of the application. Too bad, but I hope they learned a lot about debugging! 🙂

Quote of the day

Pursuit of perfection is futile.
Instead, I prioritize and often realize
goals or tasks I’ve been aiming for
just aren’t that important.”

Aisha Tyler

Reading suggestions

Not sure about uncertainty – Ministry of testing Dojo with James Thomas
When do we stop a Test? – Developsense
Coverage, ins and outs – Hiccupps

Previous posts in the series

Title and linkCategory
Part 1: IntroductionNone
Part 2: Mischievous MisconceptionsTrap
Part 3: The RiftWeapon
Part 4: The FascadeTool
Part 5: The Temptress’ TrailsTrap
Part 6: AlliesWeapon
Part 7: Don’t turn backTool
Part 8: The GluttonTrap
Part 9: Beyond the borderWeapon
Part 10: Forever and neverTool
Part 11: The ShallowsTrap
Part 12: The TwinsWeapon
Part 13: The ObserverTool
Part 14: AlethephobiaTrap
Part 15: Opus interruptusWeapon
Part 16: The IllusionistTool
Part 17: Fools’ assumptionsTrap
Part 18: The UnexpectedWeapon
Part 19: Constantly ConsistentTool