If You’re Worried About People Using ChatGPT in Your Tech Interview
I’ll show you how to love the LLM and still find exceptional candidates.
As a hiring manager, I get a bunch of (unsolicited) content from the likes of Hackerrank and Leetcode in my email telling me with the advent of Large Learning Models (LLMs) being able to accurately produce working code based on some requirements, Strong proctoring tools and plagiarism detection systems have become essential, and can help protect even solvable questions. More often than not however, these proctoring tools add little value to my goal of identifying a strong candidate who adds value and is a force multiplier for the team.
Of course all this meda about ChatGPT shredding the Google interview loop and how people use it to plagiarize take home exams is just fear based marketing latching onto the current trend following ChatGPT’s launch. Because of this bad press, leaders think the floodgates open and exceptional engineers are indistinguishable from the ones only taking a single HTML and demanding a top-tier FAANG role. While this may have some truth to it, the reality is this is bound to happen when you see tech interviewing as a game to be played.
There are two things leaders need to do to be able to confidently shrug off any fears of GPT-enabled cheating and plagiarism.
Leaders Should Interview for Substance
Interviewing for substance isn’t the “humanities” portion of the exam. It’s a conversation about the candidate’s past. You can scan a few resumes and find bits that are interesting to you and relevant to the job, set up some coffee chat sessions and dive into those stories. You can even use ChatGPT as an ally here — asking it to summarize the resume in the context of the job description to come up with some ideas (not the questions themselves) about what to dig into. Remember, this is not a standard issue behavior round; if you ask canned questions, you’ll get canned answers.
In this conversation you should learn about the candidate’s character. Are they the kind of person who recognizes and takes on problems autonomously? Do they ask for help when stuck or just sit and spin? Do their stories line up with how their resume is presented or is this a ChatGPT resume? At the end of the day, having a conversation with the real person in the moment is the shortest path to separating computer generated bullshit from the real character of the candidate.
Train For Interviewing for Substance
Most engineers I meet are woefully unprepared to interview candidates. If they do get any interview training, its mostly compliance-driven bias and legal training or training on what kinds of filter questions to ask and the pass/fail criteria. So many companies ask a filter question over Zoom just for the “on-site” to be more of the same type of questions over Zoom… so what’s the point? Nobody knows engineers better than engineers, so they should have a deeper technical conversation about past experience, swapping war stories and mapping that onto the company problem domain… not a rehash of finding the optimal N-Queens placement.
Ideally engineers should know how to suss out, interest in the problem, how they would get along, and how the candidate approaches problems, regardless of whether they arrive at the right answer in 45 minutes (as long as you’re on the right path, you’re OK in my book). Your engineers could even retrospect the filter portion with the candidate and during that conversation you’ll know with the right training and sensibilities if the candidate is genuine or submitting ChatGPT code.
A note about tiered interview loops: Some companies like to have the candidate interview the team, manager, skip manager, and sometimes a VP+. This, to me, is indicative of a micromanagement culture where senior leaders are not training their junior & mid level leaders in team building and talent identification. If the manager and VP are asking the same types of interview questions this whole chain is redundant. Leaders should always be training in team building at all levels.
Wrapping Up
Seasoned leaders will recognize this as a no-brainer and I wouldn’t blame you for bailing on this article here. It generally boils down to having an actual conversation or two with the person you’re bringing onto the team. Filters are great when you’re a FAANG and everyone wants a piece of you, but that shouldn’t be all there is. If you follow these tips, you’ll quickly weed out the bullshitters and find developers faster than churning through a ton of great developers in a futile search for the perfect one as evaluated by a one-dimensional code exam.
No, this article was not written by ChatGPT 😉
