Apart from my workshop, I also got to do a presentation at CAST 2018 – Recruiting for Potential.
If you are only looking for the slides, or want to check the slides before/after you read this post, they can be found here, along with a few of our cases.
This started out with the fact that I have grown uncomfortable with the standard recruiting process* over the years. No, let’s be honest: I’ve always disliked it, but I thought it was only me. The more I investigated it I realized it was not just my total inability of reading between lines and asking relevant follow-up questions, the whole process is biased and does nothing to promote diversity or increase the chance of hiring the best candidate.
Some of the things I noticed are kind of obvious, such as that confident, calm and positive people tended to have easier
interviews and more often get call-backs. Especially people a bit too senior for the position (the safe bet) and people that were a lot like the interviewing person. Of course, there is a big impact coming from racism, sexism and all of the other horrible prejudice we have but there are so many other, more subtle biases involved!
I started reading up on bias and found a rather intimidating video speaking of how resumes, interviews, references and
all the zillion tests we use have very little actual correlation to finding the right candidate. Our biases own us and until we acknowledge them, we won’t ever get past them. I can’t find the original video, but this is a great short one speaking about bias: https://www.youtube.com/watch?v=Q0ne13lv8hE
To try and get a bit fairer in my assessment and try and find the potential great tester rather than the person with the fanciest resume and best interviewing skill I decided to include a practical exercise as a part of the process. This is standard when recruiting for some roles, but I had never done it and I wanted to do it well.
One thing I didn’t want to do was an unprepared, on-site assignment. I know a lot of people like it but I think it (also) favours extroverts over introverts and frankly, I’m not a good enough coach yet to make it a good experience for the candidate. So, I decided to send out an assignment in advance and then discuss the results at the interview.
Next thing was that I wanted to look at more than “just” what tests the candidate could come up with. So, the case looks at multiple perspectives and parts that we think are important: analysis, planning, execution, storytelling and reporting.
We set up an application, created a background story to our fictional business and wrote a specification with a number of requirements.
Iteration number one looked something like this this.
It was an amazing experience! I found out so much about my own collection of biases and how important (and hard!) it is to find true passion in a person. We used this several times but as I grew more aware of bias and diversity issues, I started to see problems with it. The main one being that it took way too much time for the candidates. It doesn’t matter
that you can solve it in a few hours and that we specifically told them to, people want to do a good job and spent way, way too much time. It had also never occurred to me that people might not have the time or equipment to do the exercise. They might not be able to afford it. My main takeaway is this:
- Respect other people.
- Make sure you pay for their time or make it no longer than a normal interview would be.
- Offer to provide whatever equipment they might needto complete the exercise.
Or: be very aware that you are pushing some candidates out the door.
This first case made for some amazing discussions though, and I still use it for other purposes such as training.
We adapted it for whatever role we were recruiting for (such as business analyst/requirement analyst, test lead etc)
but the basic format looked the same. We had some unexpected reactions, such as candidates cancelling interviews because they thought the case did not match the role, but all-in-all it has been a really positive experience.
The latest iterations have tried to narrow and deepen the scope. When we were looking to recruit a tester to a team where it was important that the person has good coding skills and a deep technical knowledge we tried to ask a few specific questions that were meant to look for not only if the candidate can find bugs but also
- Do they do shallow or deep test?
- Can they tell a story?
- If they get the code, can they read it? Suggest a fix?
- Are they comfortable using tools?
- Can they suggest a good test for this bug fix?
The case can be found here
In that interview we spoke a lot about what the candidates thought about things like automation, security testing, model
based testing, load- and performance testing and the concept of “Automate all the things!”
This is my favourite so far and a format that I think we will stick to in future processes. Of course which questions we
ask will differ with what is important to the team then and there (and what we see in the close future).
As CAST, the Q&A afterwards raised some interesting ideas and among those I will try to implement at least one: Giving
the candidate the option of home- or onsite assignment. It shames me that I never thought of this myself but that’s bias for you! I hate onsite assignments and did not see that my preferred way might actually be a nightmare for someone
else.
*NOTE: The standard process I am referring to might not look the same in your country/culture but in Sweden it looks
something like this: a recruiter or manager scans resumes. Then follows a number of interviews, how many depends on the company, and then often some kind of test(s). The tests might look at things like personality, motivation,
performance or even IQ. As a last step the companies often contact a few references.
