So, I thought I’d write up a post about the
topic of my presentation at CAST 2018: Recruiting for Potential.
First of all, this will be a long post. For
those of you that just want the slides, or want to check the slides
before/after you read this post, they can be found here, along with a few of
our cases: http://bit.ly/recruitingforpotential_all
Basically, I have grown uncomfortable with
the standard recruiting process* over the years. No, let’s be honest: I’ve
always disliked it, but I thought it was only me. The more I investigated it I
realized it was not just my total inability of reading between lines and asking
relevant follow-up questions, the whole process is biased and does nothing to
promote diversity or increase the chance of hiring the best candidate.
Some of the things I noticed are kind of
obvious, such as that confident, calm and positive people tended to have easier
interviews and more often get call-backs. Especially people a bit too senior
for the position (the safe bet) and people that were a lot like the
interviewing person. Of course, there is a big impact coming from racism,
sexism and all of the other horrible prejudice we have but there are so many
other, more subtle biases involved!
I started reading up on bias and found a
rather intimidating video speaking of how resumes, interviews, references and
all the zillion tests we use have very little actual correlation to finding the
right candidate. Our biases own us and until we acknowledge them, we won’t ever
get past them. I can’t find the original video, but this is a great short one
speaking about bias: https://www.youtube.com/watch?v=Q0ne13lv8hE
To try and get a bit fairer in my
assessment and try and find the potential great tester rather than the person
with the fanciest resume and best interviewing skill I decided to include a
practical exercise as a part of the process. This is standard when recruiting
for some roles, but I had never done it and I wanted to do it well.
One thing I didn’t want to do was an
unprepared, on-site assignment. I know a lot of people like it but I think it
(also) favours extroverts over introverts and frankly, I’m not a good enough
coach yet to make it a good experience for the candidate. So, I decided to send
out an assignment in advance and then discuss the results at the interview.
Next thing was that I wanted to look at
more than “just” what tests the candidate could come up with. So, the case
looks at multiple perspectives and parts that we think are important: analysis,
planning, execution, storytelling and reporting.
We set up an application, created a
background story to our fictional business and wrote a specification with a
number of requirements.
Iteration number one looked something like
this: http://bit.ly/CaseTester
It was an amazing experience! I found out
so much about my own collection of biases and how important (and hard!) it is
to find true passion in a person. We used this several times but as I grew more
aware of bias and diversity issues, I started to see problems with it. The main
one being that it took way too much time for the candidates. It doesn’t matter
that you can solve it in a few hours and that we specifically told them to,
people want to do a good job and spent way, way too much time. It had also never
occurred to me that people might not have the time or equipment to do the
exercise. They might not be able to afford it. My main takeaway is this: Respect
other people. Make sure you pay for their time or make it no longer than a
normal interview would be. Offer to provide whatever equipment they might need
to complete the exercise.
Or: be very aware that you are pushing some
candidates out the door.
This first case made for some amazing
discussions though, and I still use it for other purposes such as training.
We adapted it for whatever role we were
recruiting for (such as business analyst/requirement analyst, test lead etc)
but the basic format looked the same. We had some unexpected reactions, such as
candidates cancelling interviews because they thought the case did not match
the role, but all-in-all it has been a really positive experience.
The latest iterations have tried to narrow
and deepen the scope. When we were looking to recruit a tester to a team where
it was important that the person has good coding skills and a deep technical
knowledge we tried to ask a few specific questions that were meant to look for
not only if the candidate can find bugs but also
–
Do they do shallow or deep
test?
–
Can they tell a story?
–
If they get the code, can they
read it? Suggest a fix?
–
Are they comfortable using
tools?
–
Can they suggest a good test
for this bug fix?
The case can be found here: http://bit.ly/TechTester
In that interview we spoke a lot about what
the candidates thought about things like automation, security testing, model
based testing, load- and performance testing and the concept of “Automate all
the things!”
This is my favourite so far and a format
that I think we will stick to in future processes. Of course which questions we
ask will differ with what is important to the team then and there (and what we
see in the close future).
As CAST, the Q&A afterwards raised some
interesting ideas and among those I will try to implement at least one: Giving
the candidate the option of home- or onsite assignment. It shames me that I
never thought of this myself but that’s bias for you! I hate onsite assignments
and did not see that my preferred way might actually be a nightmare for someone
else.
If you got this far: Thank you for your
time!
* The standard process I am referring to
might not looke the same in your country/culture but in Sweden it looks
something like this: a recruiter or manager scans resumes. Then follows a
number of interviews, how many depends on the company, and then often some kind
of test(s). The tests might look at things like personality, motivation,
performance or even IQ. As a last step the companies often contact a few
references.