Would Heu-risk it? Part 13: The Observer

Oh, ending this week with a good tool! We are getting into data analysis territory as you can see by the rhyme:

”We might believe we know how something will be used
Believe me, by the actual statistics you might be amused
Make sure you check what in reality occurs
And stand by to adapt to what the user actually prefers”

So, what does it mean?

This could potentially cover so many different areas, it should perhaps be at least 3-4 blog posts! I can think of at least 4 different interpretations to what this card means:
– Observability
– Monitoring
– Data analysis
– User behaviour

They are all relevant and I am no expert in either but this will be a mishmash of a little bit of them all. I have tried to put some good links in the reading suggestion for any of you who want to know more.

No matter why you do it, how you do it and when you do it – looking at the actual DATA is a great idea! In order to do that, you need to have the data and preferably know what to do with it. BUT, as with security – you can do wonders with just a little, it does not have to be perfect to be valuable.

And oh my the learning potential here!
This could potentially be anything from
– Learning how to read code ”well enough” in order to evaluate impact by the actual code changes instead of relying on someone telling you what has been changed
– Learning how to read log files in order to find issues that hide and don’t show in the system
– Learning how to use tools like Google Analytics in order to learn more about how people actually use the system
– Getting to know people in the customer support part of your company and learn about what issues the users actually have and what weird workarounds exist. Bonus: They probably classify calls so there is so much data to get here! You want to know how a certain change affected the users? They will know.
– Getting to know people in the operations part of your company and learn about how to monitor the ”hardware” (even though it might be software today…) and what weird workarounds exist there. Maybe you hear that a certain server needs to be manually re-booted every monday after an update? Hm. Hold on. Does that actually match when your automation usually always fails? BINGO!

And so on. I can go on for miles! DATA is freaking awesome!

It could of course also be just to actually observe your users, with your own eyes. By doing different kinds of user testing. This card has loads of interpretations!

Regardless if you observe ”manually” or implement tools – make sure you do observe. Don’t assume, analyse.

Story time

Well. This is not the proudest moment of my career.
A ”few” years back. Ok, many years back. We were releasing a new web site. We had originally had requirements for supporting a ton of browsers and versions but in the end we had to prioritize so we decided to support Internet Explorer from a certain version and upwards, as well as 1 Safari and 1 Firefox if I remember correctly.

We had done a lot of testing. So, so much testing. We were very happy to actually get something ready in time for the release date. A set release date, of course, with some BIG marketing campaigns forcing us to work a lot of overtime. But we released. And to be honest – I was very proud of it.
I verified the release during the weekend, it all looked good. At 8AM Monday morning our first users got to try it for the first time.

9 AM we had a crisis meeting.
As it turned out, most of our users used an older version of Internet Explorer. That version could not handle our CSS-files. So, no formatting for them, just a very very ugly page where nothing worked.

It took us a few hours to sort out, no big harm done, but we could have saved a lot of bad reputation by just…. asking!

Quote(s) of the day

”Everyone wins the closer they get to understanding production”

Charity Majors

Reading suggestions

You can tidy data – QA Hiccupps
Gear up for an eCommerse site for holiday sales – Bvahani R via Mabl
Know what – QA Hiccupps
Site reliability engineering – Google
Empowerment through Observability – Abby Bangser, ATD 2019
Observability – Charity Majors
Looking at observability – QA Hiccupps
Observability workshop – gitHub repo
Observability, a 3 year retrospective – Charity Majors
Learning about observability – Honeycomb

Previous posts in the series

Title and linkCategory
Part 1: IntroductionNone
Part 2: Mischievous MisconceptionsTrap
Part 3: The RiftWeapon
Part 4: The FascadeTool
Part 5: The Temptress’ TrailsTrap
Part 6: AlliesWeapon
Part 7: Don’t turn backTool
Part 8: The GluttonTrap
Part 9: Beyond the borderWeapon
Part 10: Forever and neverTool
Part 11: The ShallowsTrap
Part 12: The TwinsWeapon