This trap might often be imposed upon us by other people and/or time restraints. Let us look at the rhyme and then look into it a bit more :
”You say that´s an edge case, too wild to come true
A waste of your time, no need to pursue
But just because it never happened before
Does that mean it never will, can you be sure?”
So, what does it mean?
Once again we visit the land of tough decisions and balancing risk vs. waste.
Exhaustive testing is impossible.
In fact: if you disagree with me so far: stop reading. You won’t like the rest.
Still here? Ok then!
Imagine having a calculator with buttons for 0-9.
It is a very simple application. Imagine the classic windows calculator but with only a subset of the available functions and buttons.
Testing that each button works (pressing 0 actually gives you a 0, pressing 1 actually gives you a 1 etc).
Testing that you can add each of those 10 numbers to each other (adding 0+1 gives you 1, adding 0+2 gives you 2 etc).
Then you want to do that for the other operators.
Then you might want to test bigger numbers than 9.
Then you probably want to test a combination of operators.
I am pretty sure we can quite easily come up with millions of tests for this very simple application without trying very hard. And the tests we can’t come up with are endless. Don’t try it, trust me.
There is always another test, another possible combination out there. And let’s face it: most of them are probably pure waste. Unless your software will suddenly be used in a space program, you don’t have to test that it can’t handle zero gravity. Unless you truly believe your software will still be running 100 years from now, you don’t have to try to figure out what quirks computers will be up to in the future and try to test that.
We test for the circumstances that matter, for people who matter, right now and in the foreseeable future. Not for the Jetsons.
Then again:
Who thought one of the big challenges in 2020 would be to find COBOL-programmers to keep some very important applications running?
Who (except… now we know about it it seems obvious) thought in 1999 we would be scrambling to handle a new century?
Who thought the world would suddenly be in lock-down?
How could we possibly know what will be important tomorrow? Or next year? Or 20 years from now?
Sorry to tell you: no-one does. You can’t. You just have to constantly level up your risk analysis skills and do on-the-fly assessments. Sometimes they will be wrong and that is perfectly ok. Some will be wrong in that you tested ”too much” and some will be wrong in that you didn’t test enough.
This was a very lengthy introduction (sorry not sorry).
Now, about those edge cases.
To get good at the balancing act of ”How much do I test this?” you will need to learn how to, among other things:
– Find information to help with assessing how unlikely something really is and the impact of it happening
– Have healthy discussions with others to get to a common understanding/agreement on what is important enough to spend time on
– Talk to users and other stakeholders to figure out what matters to them
– Keep up to date with laws, reguations and standards that affect you
– Keep up to date with, or preferably ahead of, other software that do similar things to yours
– Keep up to date with where technology is taking us. Is there an upcoming new tech that might revolutionize stuff and things? Will it affect your software?
– Understand all of your potential business areas. Are there, as an exampla, cultural differences that could change everything?
Being a tester means being a champion of quality. Or users.
However: it also means being a team player and understanding that sometimes ”good enough” is worth it.
Fighting for every single edge case is not your job. Your job is to fight for the ones that you truly believe matter and to make people understand why you think they matter.
Story time
The best story I’ve heard about this is from the Black Box Software testing – Foundation. In true ”let’s not do waste”-mindset, I’ll reuse that one.
The Telenova Stack Failure is described on slide 256 – 264 in the following link: http://www.testingeducation.org/BBST/foundations/BBSTFoundationsNov2010.pdf
Quotes of the day
”People will use our software on really bad days, possibly on the worst days of their lives. We don’t know who just had to put down their dog, or who is worried about a diagnosis or their personal safety. But someone probably is using our software when they are severely under stress. If we assume someone is having a bad day and still needs to get things done with our software, we make it better for everyone.”
Rachel Kibler
”Since you will run out of time before running out of test cases, it is essential to use the time available as efficiently as possible”
Cem Kaner
Reading suggestions
Stress cases – Rachel Kibler
U in user – QA Hiccupps
Don’t forget the edge cases – Sarah Masud, Geeks for geeks
Design for edge cases – Asli Kimya
The ongoing revolution – Cem Kaner