Would Heu-risk it? Part 2: Mischievous Misconceptions
januari 1, 2020
The category here is trap, meaning it is something we need to beware, a mistake I have seen a lot of testers (and, to be fair, others) do over and over again. And something that can set us apart from others if we start reflecting on it and being aware of it.
To be honest, it feels a bit like cheating to start with this one, because there is already so much amazing content produced about this problem. I will get back to that later, first let us look at the rhyme:
”Did you think for a second all time is the same? Buildings don´t move, everyone has a name? Oh, all the falsehoods we learn as we grow Never believe what you know that you know”
So what does it mean?
If I would ask you to explain time to me, what would you tell me? What are some irrefutable truths that you know about time? I can imagine I might hear that every minute has 60 seconds, every hour 60 minutes and every day 24h. I might hear that we have some weird behaviour like leap-days, that some use 12h clocks (with am and pm) and some 24h. Someone might even get fancy and tell me about time zones, summer vs. winter time and perhaps even about how different countries use different formats for time, dates and such. How many would tell me there are hours that don´t exist? Or hours that exist twice? That there are more ways to count hours than 12 or 24 hours? Time is such an interesting concept and keeping track of all the ways to format dates is more than at least my brain can handle.
And the same goes for most things we believe we know.
Names. What do you know about names? Perhaps that we have first- middle and last names? Hm? Well I´m sure you know by know that is not true. How about lengths? How many applications out there have limits on minimum (Well, at least 2 seems reasonable … right?) or maximum number?
How about the fact that only women can give birth or have pregnancy related injuries? (Yes, I´m looking at you insurance application that I worked on for a very long time.
Oh, but we ALL know buildings don´t move! At least that can be guaranteed! (Sorry, what about house boats? Postcodes changing? Even countries fall and rise. Heck, the entire city of Kiruna is being moved!)
We know so many things that are actually not… true. Or at least very simplified versions of the truth because it would be too time- and effort consuming to consider the whole truth at all times.
The problem is that these assumptions can cause incredibly large problems. It can cause huge economic and scientific loss. It can cause our customers to leave us. It can cause loss of time, money, health problems or even lives if things go really wrong.
An example from my own career is from working with a system that were to be migrated from one database engine to another. We had done some very tedious ground work analyzing possible problems and we felt we had a good strategy for how to verify the data. A few problems arose due to misconceptions along the way
GUIDS GUID stands for Global Unique Identifier. They are used as a way of identifying a particular piece of data and they have a lot of benefits over other types of ids such as email, personal number/social security number. The main point of course is that the total number of possible combinations is so big that the risk of two ids being identical is almost none.
What we didn´t understand at the time was that the two database engines has different ways of generating those unique ids, meaning we went from ”almost no risk” to ”oops, pretty high risk”.
If we had not realized this during our pretty thorough testing, we would have ended up with duplicates pretty soon after release to production.
Rounding We had built our development and testing strategy around the concept of ”migrate as is”. As a part of that we had built a pretty elaborate comparison engine that would run through the system and compare calculations.
We came across a number of weird differences and decided that everything below a certain limit seemed to be related to rounding. Our first mistake here was trying to fix everything to match the old system. We spent a lot of time on this but when we actually raised the issue to our domain expert it turned out that the new calculation was a lot more correct and that this actually explained a lot of problems with the old system. So, I learned that ”as is” is not always the best, or easiest, way.
The second thing I learned was that the consequences of small shifts in rounding can add up to HUGE numbers in the end. Both because I worked with incredibly large numbers in this system, but also because a calculation can include many steps and many individual numbers that are all rounded differently.