Moishe Lettvin - What I learned doing 250 interviews at Google

I bring this piece up first because it speaks about Google’s process, which sets the stage for the status quo of technical interviews.

Moishe Lettvin used to interview for Google, and at the time of the video, interviewed for Etsy. His perspective is mostly created from the Google way of doing interviews, but he does mention at the end that “Is this the best way? Probably not.” One of his main points is that interviewing is about finding signal vs. noise; determining if a candidate will be productive at your company, fun to work with, and able to both learn and teach others on the team. He admits doing that is hard work, and likens interviewing to literary criticism rather than statistics: lots of subjectivity.

So, how can someone raise the level of the signal?

Transparency - give candidates as much information about the interview process as possible
Team effort - multiple people help round out the interview
Be prepared - organizations should teach their interviewers how to perform the interviews

The quote from this video that I love is that there is “no reason to have an air of mystery in the interview process.” He says that there is no advantage in keeping secrets about the interview process. By being open, it allows you to let candidates know that you’re trying to allow them to show you their skills.

Dan Luu - hiring lemons

Dan Luu doesn’t shy away from talking about workplace problems. A recent post in that vein is called Hiring Lemons, where he discusses and debunks part of Joel Spolsky’s “Finding Great Developers.” It doesn’t so much discuss current hiring practices, but more so makes observations about the experiences of candidates. My notes are mostly interesting quotations.

In his experience, Luu has found that companies rarely understand the value of their great employees. “When I worked at a small company, we regularly hired great engineers from big companies that were too clueless to know what kind of talent they had.”
Teams that have openings are more likely to have dysfunction. Teams that are really good have low churn, and thus, rarely have openings.
- My wife was able to relate to that: She had left a job despite knowing there were some very smart people around her, but the work was not interesting, and there was so much churn that she knew the project would not attract other people that she could learn from.

Matasano - hiring

In Dan Luu’s article, he mentions “Matasano famously solved their hiring problem by using a different set of filters and getting a different set of people.” Googling “Matasano Hiring” takes us to their blogpost about their hiring process. Sections 1 and 2 have some background on hiring and their hiring needs, but the good stuff starts in Section 3.

Section 3

“Next, interviewers make up their own interviews. This is crazy.”
“It gets worse. We’re unclear about selection criteria.”

Section 4

“Engineering teams are not infantry squads. They aren’t selected for their ability to perform under unnatural stress. But that’s what most interview processes demand”

Section 5

“Confidence bias selects for candidates who are good at interviewing.”
“For every genuinely competent and effective developer who can ace a tough dev interview, there are many more genuinely competent and effective developers who can’t.”

At this point, you should really just read Sections 6 through 9 for their take on How can we interview better?, but here are my short notes.

Section 6 - Warm up your candidates

“Not knowing what to expect makes candidates nervous. That’s a pointless handicap.” This echoes the comment from Lettvin above
“We worked from the assumption that a candidate’s resume, background, and even their previous experience had no bearing on their ability to perform the difficult and specialized work we did.”
- This one is pretty intriguing. They go on to say that they purchased $80 worth of books and sent them to candidates to study up before the interview.
- This is complemented by the statement that most candidates did not know Matasano’s domain very well at all when starting the interview process, but this still resulted in great hires

Section 7 - Build work-sample tests

“Your goal is to collect data you can use for apples-apples comparisons, which means every candidate has to be working on the same problems.”
“This [work-sample test they wrote] is a couple hundred lines of code, written in a few hours. It out-predicts any interview we’ve ever done.”
This is a really great section on how you can use work samples to gather data about candidates in a repeatable, objective way.

Section 8 - Standardize and discount interviews

They talk about having interviewers read from a script, despite knowing they will dislike it.
“Interviewers hate it. But we kept doing it, because we found that we were comfortable with “why” we were making hire/no-hire decisions when we had facts to look at.”
“Phone screens carry almost every liability of interviews but further hamstring the candidate and the interviewer by occurring at a distance, being hastily scheduled, and, for the hiring team, having implicitly lower stakes than the in-person interview.”

Section 9 - Ask yourself questions about your interview process

Nothing to add here, just ask yourself the questions in this section

Example - Slack’s engineering walkthrough

As Lettvin and Matasano explained, there is no advantage in keeping the interview process a secret. Slack has taken that principle to heart and wrote a great blog post in May 2016 about their hiring process. It works as a great template for implementing this at your own company.

What next?

I want to keep adding to the reading list each time I find another interesting article in this space. This is by no means an exhaustive list, just the articles I’ve had time to read and take notes on.

Have another hiring article you love that I missed? Have critical thoughts on these or my analysis? Let me know, I always want to try improving, especially on a subject I know so little about.