Interview Coder is an invisible AI for technical interviews

> These "invisible AI" tools aren't destroying technical interviews - they're just exposing how broken they already were.

Whenever the topic about how broken tech interviews are has showed up on HN in the past, there were usually two crowds: the "they suck, but they're the best we've got" people, and the much less common "they suck, so we do something else" crowd. Almost everyone agreed they were broken.

What does it say about the tech industry that so many orgs continued to use a system that was known to be broken for so long? How much inefficiency and waste over the past couple of decades is attributable to bad hires? And conversely, how many efficiency improvements in the near future are going to get attributed to AI tech rather than the side effect of improved interviewing practices meant to combat AI candidates?

> The problem is that our industry clings to this bizarre ritual where we test for skills that are completely orthogonal to the actual job.

I agree. And it's not a good sign for the industry when engineers have to spend months working up to being good at answering these questions just to get the job.

When I think of the hundreds of hours I’ve devoted to learning how to solve leetcode problems that could’ve otherwise been spent on learning tools, technologies, and architecture patterns that would be useful for my actual job, I genuinely feel a deep, deep sorrow.

That trend was always comical to me. I’ve never been one to memorize documentation, specific function call orders, etc, and I remember miserably failing a coding test for Comcast. The test? Write a full contact management application on an isolated computer, no IDE, using notepad and no installed runtime.

You’re spot on with the uselessness of modern algorithmic questions. We live in a world of high level languages; in 20+ years, I’ve never implemented a single one of those coding problems in an actual codebase. I would prefer to hire an intelligent, sociable person with core skills and who has room to learn and grow, rather than someone who can ace silly algorithms but can’t work with a team or complex codebase.

Heck, the person you refer to having hired, and presumably fired, may not have even used these tools. The trend these days is to learn to the coding tests from sites we all know about, not the ideas or concepts needed to work in a real team on a real application. They spend all their time writing algorithms, but often never actually touch a database/write a real complex application. The bootcamp explosion was a self-escalating problem for algorithm-based test questions.

How did you know he used tools like this one? You sit him down and interrogate him? Or did they own up to it? What was the result? Did you cut them loose?

When did you switch to pair programming? Your post starts by stating you just hired the unqualified dev last month, so did this unqualified dev slip through your pair programming system? And if they did, doesn’t that undermine your view that the pair programming approach has dramatically improved your good hire hit rate (or, if pair programming is newly instituted in response to the unqualified dev hire, how can you claim to have an improved hit rate in only one month of use)?

> My current team scrapped the algorithmic questions entirely. We now do pair programming on a small feature in our actual codebase, with full access to Google/docs/AI. The only restriction is we watch how they work. This approach has dramatically improved our hit rate on good hires.

I've been on this gig for about 20 years, this process is by far the best one to participate in an interview both as an interviewee as an interviewer.

As an interviewer I have a lot more joy to tackle an actual task, or a problem very similar to how my team works, with someone applying for the position, it's easier for me to feel like a human instead of an assessment machine. I don't have to learn the N different potential case studies to run the interview, I don't need to feel like I'm ticking boxes when giving my feedback, it's all natural.

As a candidate it relaxes me a fucking lot, I don't have to brush off old books, go through a grind of studying and still feeling like I haven't studied enough because there's an endless amount of knowledge to learn if I need to cover all bases. I also feel a lot more like a human, I can talk through my train of thought, work as I normally would checking documentation, searching, etc.

This bizarre ritual feels almost like hazing by this point, at some point Big Tech decided this was "the way for hiring", then herd mentality took over to the point where tiny startups just cargo-culted processes without even questioning (I heard from a non-technical founder once "if Google does it then it's the best way to do it"). A few generations later of folks hired through this Byzantine process turned it into a hazing ritual, if they had to go through this pain then they might as well inflict the pain to make candidates prove themselves worthy.

It's a giant cycle of bullshit, and no matter how much I tried it's been always impossible to change the minds of HR that this is not the best way to assess candidates, the herd mentality is even stronger around there since there's always the dumb scapegoat: "we do benchmark peers in the industry and they all do this way, so we will do this way".

Sorry for the rant but it's one aspect of this job that really grinds my gears, I fucking hate running interviews with candidates because of it, I never feel I could properly measure someone's real abilities, and I really hate having to go through it myself...

It’s a major improvement but you still want to be careful on what you consider common knowledge. There’s a lot of breadth to software engineering and there can be whole areas someone is missing but can learn.

For example I was always a great employee but early in my career I wasn’t big on unit testing.

Or I interviewed for an ML job and they dinged me for not knowing a bunch of statistics things off the top of my head.

This is a great approach and I like it as well. I did have a few situations where the person could talk shop like an expert but when it came to actually writing some code they failed. I literally had a person fail fizzbuzz that was supposedly senior and talked shop really well.

This doesn't scale, and opens up legal liability to unfair hiring accusations.

Is it fraud when they tell you you’ll spend most of your time coding cool new products but then you mostly do customer support and meetings?

Or maybe it's defensive against an industry who has developed ridiculous hiring practices.

> a tool allowing job candidates to commit fraud

It's not fraud.

The candidate could also use this tool to help with the job once they've been hired, and that would not be fraud either.

They wouldn't, but that's just because the interview is stupid and not like real work.

Pretty much every company I've applied to or worked at commits both legal and moral fraud against its employees and applicants, on an industrial scale. They've also created a broken interview process that punishes honesty and forces applicants to find creative solutions. So personally I see no moral fraud here.

It is okay to cheat when the game is rigged. At least this is what star trek has taught me (Kobayashi Maru).

You're not wrong, but you're overlooking the cultural normalcy. When you have something to sell, you're allowed to lie until you're blue in the face and nobody even blinks. But when applicants merely demonstrate how well they understood and internalized that state of affairs, it's fraud. None of this is OK in my book but it's hypocritical to single out job applicants when the whole culture is like that.

In most societies in the world today you must have a job to survive and to support the survival of one’s family. Imho it is not morally wrong to do anything you need to do to achieve gainful employment so that you and your family can survive, and I would go so far as to say it is immoral to be scolding people engaged in a fight for survival that they aren’t doing it properly.

> System Design interviews

System Design interviews can be crammed too.

I'd find it much better if people would ask you "tell me about some complicated systems you helped build and what high-level challenged you encountered". Instead you get "how would you design google docs?".

Does “in-person” mean flying in the candidate, and performing the interview on-site?

Does “100% of their loops” mean everything after the initial contact?

I suspect smaller companies could find this challenging.

I’m old enough to remember when independent recruiters acted almost like “talent agents,” with large payouts.

This encouraged them to curate a “personal brand,” and they would often do a lot of the vetting, themselves.

I believe that executive recruiters still operate that way, but engineering recruiters seem to have sharply declined, since those days.

Don't forget about implementing quicksort on a whiteboard.

I believe people who are using these AI tools to pass interviews wont be able to use AI in their real job in a net positive manner.

I couldn't agree more. LLMs are legitimate tools and, ideally, I want to see how effective a candidate is in using their available tools to solve complex problems.

The service on offer here is different. It's providing a means to use LLMs to fake your way through a technical interview.

Showing that you can use LLMs to quickly and correctly solve problems is a good skill to have. Offering up a solution from an LLM as your own work without acknowledging how you got there is just misrepresentation... or to put it another way is just lying. Maybe fake your degrees and experience while you're at it, right?

At least in the long run, many that need these tools to get in will be found out once they start having to solve real problems on the job. Just a shame about other, more qualified people being turned away. Of course if the LLM was sufficient enough on its own, perhaps a real software developer was never required to begin with.

Something very like that was a Launch HN in, I think, 2021. "Skip the Interview" was the name, I believe. But no doubt with that example to learn from it will go at least a little better this time.

And why would I pay to refer someone? If a company suggested that I'd assume it was a scam, and even if they were well established I don't see an incentive that isn't worse.

What is your interview process then?