You’re terrible at hiring — A startup’s guide to interviewing engineers

I frequently get asked by startups for help hiring engineers. Everyone wants a quick solution, but the problem with hiring humans is that they’re human. Studies also show that we’re terrible at picking the best candidate. Rather than throwing our hands in the air and hoping for the best, it’s even more important to have a process and to understand what it can and can’t do. Hiring doesn’t have to be hard, but that means accepting that we can’t predict the future.

As engineers, we love problems. And hiring is a problem — a damn hard problem. One that people like Lazlo Bock of Google have spent their careers trying to solve. When asked about how Google performed at this task, he had the following to say:

“Years ago, we did a study to determine whether anyone at Google is particularly good at hiring,” Bock says. “We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job. We found zero relationship.”

Zero relationship.

“But I’m different”

I bet you’ve got an opinion on the “right” way to interview. Maybe you’ve interviewed hundreds of candidates, or maybe you’ve been on the receiving end of a few bad interviews. You hate coding questions. Or you love coding questions. You like to whiteboard. Or you think whiteboarding is a waste of time. You can determine the height of a binary tree. Or you feel that algorithm questions don’t equate to job performance.

But the evidence speaks for itself. One of the largest and most successful tech company in the world has determined that there is “zero relationship” between interviewers assessments and how candidates ultimately performed. No one was good at it at during tens of thousands of interviews at Google, so it’s improbable that you’re any different.

So what’s the point of interviewing?

It would be a mistake to look at Google’s data and conclude that interviewing is a waste of time. Google said there was no correlation between interviewer assessment and success, not that the interviews themselves were useless.

Interviews serve two purposes:

1. To gauge whether a candidate is capable of performing the role for which they are interviewing. This is not the same as job success. Plenty of competent engineers fail to make the transition to a productive team member, but it’s impossible to be productive if you simply can’t do the job.
2. To sell the candidate on the company. People, especially those of us in startups, fail to realize that a candidate’s interview is likely to be there only interaction with the organization and team. The impression set here is a large part of what makes them take the job offer (hopefully it’s not just the compensation package).

For me, once I focused on those two goals, interviewing became a lot easier and less stressful.

Good interviews have structure

It’s easy, especially for startups, to play loose with the format of an interview. Even in larger orgs, I’ve been sent in to interview candidates with only a few minutes warning and no format to follow. This results in a subpar experience for both the hiring manager and the person being interviewed. “Did I ask the same questions as the person before me? What should I focus on for the role being considered?”

Improving the interview structure means getting a better idea of the capabilities of the candidate. And interviewers that know what’s expected of them, will appear more professional to the person being interviewed (increasing the chance of an offer acceptance).

My rules are as follows:

• Have each interviewing team member prepare their questions beforehand. Nothing ‘on-the-fly’
• Know who is asking what questions.
• Everyone should have some background on the candidate. “Uhhhh, so let me look at your resume” is never a good look for an interviewer
• Avoid ambiguous questions. “Tell me about yourself” leads to either confirmation bias, where we reinforce our first impression of the person or meaningless realizations like “OMG, we lived in the same dorm”. Neither of these tells me whether or not the engineer can do the job at hand.
• Questions need a solution. It’s OK to have an open-ended discussion but understand what constitutes a wrong and a right answer. Remember, we are looking for capabilities, not trying to “figure out how they think”.
• Focus on determining general engineering competency. Can they show you something they’ve worked on? Can they take something home to solve? Not everyone does well in a coding interview. Don’t miss out on a candidate because they get nervous coding with someone looking over their shoulder.
• Take notes. Every interviewer should be taking notes during the interview and finalize them afterward. You want to get this information while it’s fresh and not two days later.
• There is no perfect question or way to interview. Each member of your interview team is going to have their own idea of what constitutes a good question. Some will want to code, and others will focus on algorithm questions. That’s OK. Just make sure that they understand that the goal isn’t to pick the right candidate, but to discover which people that are capable of doing the job.

Interviews that close candidates

Tech companies are notorious for asking brain teasers. Although Google claims to have curbed the practice, stating that they “serve primarily to make the interviewer feel clever and self-satisfied”, they’re still well known for asking “hard CS” questions during an interview.

I used to have a negative opinion of these types of questions. I don’t think they give you much insight into a candidates abilities, but I’ve come to appreciate them for a different reason. Not only do they provide a way to make the candidate feel smart when they answer it correctly, but they also to make the interview seem “tough”. The fact is, people perceive things they worked hard for as more valuable. Case in point, every Google engineer will happily tell you how hard their interview was. While you may find it annoying, it’s a testament to how a challenging interview shapes a candidates view of the company and the prestige of the role. Interviews are a form of selling, and hard interviews close candidates.

But, Culture Fit

Sure, it matters if the person you’re hiring is a huge asshole. Outside of that, stop trying to be an organizational psychologist. Look for big red flags, but don’t spend your time stressing about it. The biggest culture fit mismatch is the inability to do the work at hand.

Stop trying to predict the future

There’s no one size fits all solution to hiring for startups. Each startup and each role should have a different process tweaked for their own needs. That being said, it’s critical to understand that when it comes to humans, you’re never going build a perfect process. Instead, stop trying to predict the future and focus on finding a candidate that has the ability to succeed.