Large tech companies like MAANG use live-coding interviews to filter out potential bad hires. Does it work though?

Short answer

No.

The longer answer

Making a live coding session part of your interview process has shown to yield a high rate of false negatives as well as seeing talent leaders turn away without giving you a chance to interview them.

Why the false negatives?

Pressure

Many people can perform well under pressure, but I doubt you’ll ever get a candidate to show you their utmost best when placed under such pressure.

It could be that you’re labouring under the illusion that they will, or you’re admitting that you cannot provide your employees with a safe environment where they will perform at their utmost best.

The Script

I’ve seen this, in fact I have been guilty of this myself, the interviewer goes in with their problem and expected solution ready to go.

  • Your solution could be expecting a very particular pattern and you’re rolling the dice that the engineer has recently worked with that pattern and will be able to remember how to implement it.
  • Your solution is deliberately convoluted, but the candidate recognises an anti-pattern and suggests a more appropriate solution. If you’re lucky your interviewer will be open to hearing this solution but if they’re not, that potentially great candidate is out the door.
  • Your problem might be so mundane that the candidate simply loses interest in working for you.

Transparent skills

On any given week I work with 3 to 4 different programming languages, not to mention the scripting languages, markdowns, and the various platforms I work on. This isn’t me trying to brag, I think any engineer that builds on his craft in his free time does even more than that, my point is rather that there is a lot to retain but our biggest skill is being able to research answers, understanding when we’ll need specific answers again so we can retain them and when we won’t so we can forget about them in favour of retaining something else.

I doubt your interviewer will accept an answer of “Ah, that’s an implementation of the peer-to-peer pattern, let me Google how to best implement it again.” But the reality is, that’s likely what even your own interviewer would do for most problems in their day-to-day.

Similarly, when you ask a candidate to code something in notepad instead of their native IDE. They’ve spent years building up a skillset in using their particular IDE from shortcuts to macros.

My point here is, you’re forgetting the supporting skills that potentially make them a great candidate.

People skills

Does your Talent Specialist code? Maybe.
Does your specialist engineer have great people skills? Maybe.

A live coding session is already a pressure filled exercise, adding more than one interviewer into the mix won’t help that so you might have to sacrifice one of the above and hope it all still works out.

Why are people turning away?

This one you could answer by doing a simple web search for “Live Coding Interviews” and glossing over the many programmer forums out there.

The gist of it is that programmers have begun to associate live coding interviews with poor culture and as such companies that aren’t worth their time.

Why does MAANG do it then?

How many responses do you get for engineering postings when they go live? LinkedIn Easy Apply doesn’t count!

Tens? Maybe even into the hundreds?

Meta, as an example, will get upwards of thousands, in many cases tens of thousands. A large portion of these will be people hoping to get a free ride, some will simply not be at the level that the role is advertised for, etc.

Meta can “afford” to have a high false-negative rate more than they can afford the above. That’s not to say they should be doing it, there are other alternatives, but this has been the predominant way for much of MAANG to filter out bad eggs out of the thousands of applicants they get.

What are the alternatives?

There are many alternatives, I won’t be able to list them all here but how about one or two suggestions?

Pre-determined exercises

This is an easy one, send out a skill-based problem (the same one you would likely have asked the candidate to do live) and ask them to submit the solution at least a few days before the actual interview.

But what if they just used GPT to provide the solution? It’s important to note that you’re not trying to catch the candidate out, that should never be your objective and if it is, you should feel bad. You’re trying to gauge their understanding of the solution.

By submitting it a few days in advance your interviewer can immediately assess if someone was unable to get to the right solution and you’ll waste no further time. If they were able to get the right solution, your interviewer can query them on it with innocuous questions like:

  • What made you decide to do it this way?
  • Would you consider changing it around this way? Why not?
  • Do you know what kind of a pattern you applied in this case?

All these questions open up more questions which make it quite easy to determine if a person was lying when they submitted something or whether they genuinely understood what they were doing.

I like this approach because if there is one thing I’ve noticed about the best coders out there it’s that they’re almost always passionate about what they do, and you can quickly tell when you ask them about their code how passionate they are. It also removes the pressure of a live exercise and creates a much more flowing conversation.

Online test kits

These exist to give candidates a platform on which to provide answers to various questions as well as a couple of solutions based on problems. Often you can create your own custom tests to tailor them to your specific skill requirements.

Examples of these platforms would be TestDome and eSkill.

But what if they just used GPT to provide the solution? Same answer applies, make sure to furbish your interviewer with the answers provided by the candidate and question them on that. You’ll learn more about the candidate when having this sort of a conversation than when you have them under pressure doing a live exercise.

Conclusion

I’ve pushed back against live coding exercises because I’ve seen incredibly smart engineers fold under the pressure of an interview. Maybe it’s a risk you’re willing to take in your organisation but know that there are better ways of accomplishing the same goal, why would you?