On the Other Side of the Barricade: Job Interviewer Do’s and Don’ts

Disclaimer: as usual, the opinions within this article are those of ‘No Bugs’ Bunny, and do not necessarily coincide with the opinions of the translator or the Overload editor. Please also keep in mind that translation difficulties from Lapine (like those described in [ Loganberry04 ]) might have prevented providing an exact translation. In addition, both the translators and Overload expressly disclaim all responsibility from any action or inaction resulting from reading this article.

Finally, all the years of hard work have paid off, and you’ve reached a team lead position (which should have been much easier if you’d followed all the advice contained in numerous ‘No Bugs’ articles ☺). One of the responsibilities which often comes with such a position is conducting technical interviews with job applicants who have passed initial resumé screening and are considered as candidates to join your team. As with many things out there, interviewing is easy to do wrong, and difficult to do right. In short, it is a skill, and an important one (especially if you want your team to perform). Of course, you’ve been on the other side of the interviewing table dozens of times, but on this side it is very different. So, what can be said about conducting job interviews?

Task definition

First of all, let’s try to describe what we’re trying to achieve. Of course, the task is ‘to tell if the candidate is fit for the job’, but it would be great to translate it into something more tangible. Do we really want him to know the API by heart or is just finding stuff she needs, within 30 seconds on the Internet enough? Do we want him to understand how the things are working, or is the ‘black-box’ approach is good enough? Do we want her to be a quick learner (because our project will take some time to grasp), or we want him to be able to work right away using technology XYZ?

Answers to many such questions depend a lot on the specifics of the project, and on the culture within the team. However, there are several things which interviewer needs to aim for in many cases regardless of specific project:

• we need to find out if the candidate is a ‘thinking’ developer or ‘following-instructions’ one (there are positions requiring both types, but placing one type into position which requires another type, is not a good idea)
• we would like to learn if the candidate has an ability to grasp things quickly (it is often very important, especially in agile development environments)
• knowledge of fine details of APIs (systems/protocols/...) by heart is normally not required, so this is what we shouldn’t concentrate on; what is usually much more important is ability to find things a developer needs quickly.
• understanding of APIs (systems/protocols/...) is a very different beast, however, if you need somebody to start working quickly (without a learning curve). It does require an understanding of concepts which are relied on in the API/system/protocol, so this is one thing we need to work on during the interview.
  • To illustrate the difference between ‘knowledge’ and ‘understanding’, let’s assume that you need to hire somebody to work on a Windows client. To do the job, usually the candidate doesn’t need to be able to answer questions like “what is parameter #14 of the CreateFont() function” (it takes 15 seconds or so to find it in MSDN); however, an understanding of the concept of windows messages, threads where these messages are processed, and sometimes implicit priorities associated with some of these messages can be useful in quite a few projects.
  • and last but not least, we’d like to make sure that good candidate would still want to work in your team after the interview (remember, any interview works both ways – you’re assessing the candidate, and she assesses your team based on your actions during the interview)

While this list is by no means exhaustive, I feel that it is a reasonably good starting point to prepare your own list of goals.

Now let’s proceed with some do’s and don’ts for the job interview (keeping in mind our goals stated above).

Do #-1: Do have a resumé in front of you

It should really go without saying, but you do need a resumé in front of you during the interview. Not only you will need it (you’ll see below why), but also not having it (as well asking candidate if he has a copy of his resumé) creates an impression of being unprofessional, which is one thing to avoid (remember that point about candidate assessing your team based on the interview?)

Do #0: Do prepare for your first interview

Conducting job interviews is not that easy and not as fun as it might look when you’re sitting on the other side of the table. For your first interview to be meaningful, spend some time, read the candidate’s resumé, think about things you would like to know, and about questions you’re going to ask. After a few interviews, you’ll be able to conduct job interviews without (much) preparation, but for the first one – do yourself (and the interviewee) a favour and come prepared.

Don’t #1: Don’t try to show you’re better than the candidate

I’m important
I’m really important
Did I say I’m important?
I’m really important

~ ‘No Bugs’ mantra for those suffering from an inferiority complex

After doing the very basic homework described above, the very first trap a new team lead can fall into when interviewing is to try asserting that he’s better than the candidate he’s interviewing. It is a Very Bad Thing™ for several reasons, with the main one being “if you’re trying to show you’re better, you’re not doing your job of interviewing”. However, I’ve personally seen this kind of behaviour many times, and it is so common that I simply must caution team leads against it. The main symptom of this ‘boosting self-esteem via demeaning others’ syndrome is asking questions nobody of a sane mind can possibly answer (unless she accidentally ran into it within last 2 weeks, as happened to the interviewer), or saying that the candidate is wrong not because of substance, but because of terminology (which is always easy when you’re interviewing). If you ever find yourself asking such impossible questions or arguing about terminology, you need to sit back and ask yourself, what is the real reason you are doing it?

Those suffering from an inferiority complex (and trying to boost their self-esteem by showing they’re better than at least somebody), may console themselves with the thought that the mere fact that it is you interviewing automatically makes you better. While this logic is fundamentally flawed, it is still much better than conducting a job interview with the sole intent to show that the candidate is not on par with (usually the intent is more like showing that the candidate is ‘light years behind’) the interviewer.

Don't #2: Don’t ask questions about the 14th parameter

The second worst thing which can happen during the interview is when the interviewer starts to ask questions about subtle details which nobody really cares about and which can be found in man (MSDN, on the web, etc.) within 30 seconds. One extreme example of such a question is an aforementioned question about the 14th parameter in the CreateFont() function. In defense of this approach, I’ve heard arguments like “if he doesn’t know this, why are we even talking?”, and I’m sure that both these lines of argument and the questions are deadly wrong. Such questions (especially if accompanied with ‘doesn’t even know this’ logic) often stem from the interviewer working for a long while within one very specific development project, when he needs to remember such things just because this project forces him to do it 15 times a day. However, when interviewing, one needs to remember that people come from different environments, and that those 30 seconds spent searching MSDN will not affect developer performance in any way. Another (and probably main) reason to ask such questions is that they’re really really easy to ask (finding smarter questions requires much more thinking); however, this is one case when you get what you’ve paid for. While smarter questions are indeed much more difficult to find, they’re also much more useful for our goals.

Do #1: Do ask questions leading to a dialogue

Remember that we’re trying to find if the candidate is able to think, right? The interviewer is trying to determine how to distinguish smart candidates from not-so-smart ones (or, putting the same thing in a politically correct way, how to distinguish very smart candidates from simply smart ones)? Asking multiple-choice questions, and questions which should be answered in a single word, usually does not work well for this purpose; instead, it is much better to ask questions which start dialogue. For example, the question “what does a+++b mean?” usually is a pretty lame question (not to mention that it also falls under the ‘questions about the 14th parameter’ and probably under the ‘showing that you’re better than candidate’ categories), because it is either the candidate knows the answer, or he does not. To compare, a question like: “Can you have a static virtual function?” is much better in this regard, as for any answer (especially for the correct one) you’ll be able to ask “Could you explain why?” The answer to this ‘why?’ question will tell a lot more about the candidate than a dozen ‘ a+++b ’ questions. In practice, this ‘static virtual’ question lies on the lower end of the spectrum of questions leading to dialogue, and if the candidate is good, one may proceed to more elaborated ones like, “What is the guarantee on insertion into std::map<> ?” (once again, to be followed by “Could you explain why?”).

It is important to remember that reasoning in answers to such questions is much more important than the answer itself. For example, when answering a question about the possibility of inline virtuals, the answer “There is no such thing” is technically wrong, but if the candidate gives a good explanation why he thinks so (such as “because virtual functions should be resolved via virtual table, which won’t work for inlines”), it is much better than the simple answer “It is ok” without any evident understanding.

It is also worth noting that potential for the dialogue is heavily context-dependent. The same question about ‘ a+++b ’, if asked when interviewing for a position in compiler development, can easily be a good one, leading to a healthy dialogue about lexical parsers in general and Lex in particular.

Do #2: Do ask questions about previous projects

If the candidate has substantial previous experience, it is usually a very good idea to ask her about previous projects. It can be either specific projects on the resumé which you feel are relevant to what you’re doing (or projects that simply look interesting for any reason), or can be a very generic “Could you tell me about the project in your career which you like the most?” In fact, if the candidate is experienced (and assuming you have 15 minutes to spare, which you should), it is almost always a good idea to ask the latter question. You’ll see immediately if the candidate is excited about what she’s done in that favourite project, and you’ll also see if she’s excited about the right things. In my experience, there was a candidate rabbit who positioned himself as a senior developer, and who was really excited about the function he wrote to parse e-mail addresses (and it didn’t recognize complicated stuff like RFC822 comments and domain literals); well, in my books it didn’t really count as a “lifetime achievement to be proud of”.

One thing to remember in this regard: if candidate speaks in detail about a project but doesn’t mention his role in it, don’t forget to ask him.

Do #3: Do ask questions about things on the resumé

If the candidate has mentioned something on the resumé, they should be able to talk about it. Treat the resumé as a carte-blanche to ask about anything the candidate has mentioned. Remember – if he’s lied about one thing, he might have lied about the others. While not necessarily fatal, you (both as an interviewer and as a potential team lead) really need to know about it. So, if you see on the resumé something like “in-depth knowledge and 5 years of hands-on experience with boost::containers, such as vector, map, etc.”, don’t hesitate to ask “Could you remind me if insert() can invalidate vector iterators and why?”If this is not answered, just write it down and be much more careful with trusting any other claims in that resumé.

Do #4: Pay attention to candidate communication skills and style

While it is often not too easy for somebody coming from the development side, it is still really important to remember that the candidate is not just a machine to convert coffee into lines of code: she is also a person who you will need to work with, and communicate with. The key questions here are similar to the following: “Can I sit beside this person all day?” “Do I want to go for a drink with them?” However, to be on a safe side, it is better to double-check with your HR if you’re allowed to take this kind of things into consideration (as anti-discrimination laws and company policies can be sometimes really ugly in this regard).

Don't #3: Don’t ask questions which are too simple or too complicated

It usually happens that after several interviews. You compile a list of your favourite questions, and are following it when conducting subsequent interviews. It is perfectly fine, as long as you’re making a sanity check on your standard list depending on the candidate. If you’re asking a candidate for a position of architect something like “how much is 0x0A+0x0B?”, you’re risking that he’ll ask himself a different question “Am I sure that I want to work with team lead who asks this stupid question?” This “don’t #3” shouldn’t be interpreted as “if you’re interviewing for the position of architect, restrict your questions to ‘how to play golf?’”, but questions which do nothing but waste the time of both interviewer and candidate should be avoided.

On the other hand, one also should avoid asking questions which the candidate will not only be unable able to answer, but also will have problems comprehending. For example, if the candidate has already indicated that he has no idea what STL is, continuing to questions about guarantees for STL containers is pretty much pointless.

Neither do nor don’t: Tests

Good is better than bad
Happy is better than sad
My advice is just be nice
Good is better than bad

~ Pink Dinosaur from Garfield and Friends

On tests during the interviews, there are two common points of view: the first is that a test is the only thing that matters, the second one is that all tests during interviews are evil. As usual, I disagree with both of these ☺. As I see it, good tests are good, and bad tests are bad, but there is a BIG problem with how to write a good test.

The best test I’ve seen was a task to create a working program for a certain algorithm (the algorithm came with test vectors). The test took several hours, but the developer was allowed to use anything she needed (compiler, debugger, Internet, whatever; there was even an option “you can write it in your favourite programming language”). Some didn’t like it, but it has indeed proved to provide an extremely good insight into the developer’s style, attention to detail, code structuring, comments, etc. While it was still just an insight and was not 100% conclusive, it was extremely valuable in assessing candidates. Still, there was a major problem even with that test, and it is that the test has eventually leaked into developer community, and creating another meaningful test is really really difficult. And one more thing to remember about such long tests – they can cause substantial problems with candidates reluctant to spend that much of their time with little chance to get something out of it; in the case I’m talking about, it was somewhat mitigated by the test being a part of the second technical interview, with an understanding that company is “almost ready” to make an offer if the test is successful (and also restricting the test to the second interview was rather important to mitigate the leaks).

Another bunch of reasonably good tests was given to me by companies engaged in algorithm development. These usually involved non-trivial sorting, permutations, etc. It is not clear how useful these tests were for potential employees (my guess is that they were – to certain extent), but at least they were fun for me :-).

And the last category of tests was the stuff which shouldn’t have been asked in the first place: things like “here is a program, find a bug here” - just to figure out that the task was to find a typo causing syntax error (hint for test writer: a developer is usually using compiler, so finding typos and syntax errors is not exactly what he does for living). Such tests (presumably written merely to do some kind of test and report about it to the management) are pretty much pointless (the only thing such a test can achieve, is to successfully scare away an ‘overqualified’ candidate).

The bottom line about tests in an interview: if you have a really good one – give it, otherwise – look for a good one. Oh, and a word of caution: before using tests during the interview, ask for confirmation that using tests is ok (from management, HR, or legal) just to be on a safe side. (In some countries laws can be really tricky, and company policies are often even trickier.)

Wrapping it up

Obviously, the present article does not pretend to be a comprehensive guide on ‘how to interview IT job applicants’, but I hope it may be a reasonably good starting point to start developing your own interview methods.

Acknowledgement

Cartoon by Sergey Gordeev from Gordeev Animation Graphics, Prague.