Companies Still Running Coding Interviews Will Be the Losers of the AI Era

The man who built Claude Code hasn’t hand-written code in months. AI writes a third of Big Tech’s codebase. And yet most companies still hire data scientists by making them solve algorithm puzzles from memory, alone, unassisted, on a whiteboard. The firms clinging to this ritual aren’t just running a bad test — they’re systematically filtering out the exact people the data says will drive the next decade of performance. Here’s the evidence.

The most important sentence in software right now

In mid-2026, Boris Cherny — the Anthropic engineer who created Claude Code — told a Fortune conference audience that he hadn’t written a line of code by hand in roughly eight months. Claude Code is now 100% written by Claude Code, and he noted Anthropic’s biggest enterprise customers are trending the same direction [1]. Since adopting agentic coding internally, Anthropic’s engineering team reportedly grew 4x while output per engineer rose 200% [2]. Spotify made headlines in early 2026 saying its best developers hadn’t written a line of code since December [2]. SemiAnalysis estimates Claude Code alone now authors around 4% of all public GitHub commits [2]. Back in 2025, Microsoft’s Satya Nadella put AI’s share of his company’s code at 20–30% — strongest in Python, the home language of data science [3] — while Google’s Sundar Pichai reported “well over 30%” and climbing [4]. Anthropic CEO Dario Amodei predicted AI would soon write essentially all code; Microsoft’s CTO put it at 95% within five years [4].

Discount the executive hype however you like — the measurement is fuzzy and the timelines are self-serving [5]. The direction is not in dispute: the manual production of code is being commoditized in real time, and the most productive technical people in the industry have already stopped doing it by hand.

Now hold that fact next to this one: the standard hiring gate for data scientists is still a test of manually writing algorithmic code, from memory, without tools, under surveillance, on a clock.

We are hiring for 2015’s job with 2015’s test, while the people who will define 2030 wouldn’t pass it — and increasingly won’t even sit for it.

The test was already broken before AI

The uncomfortable part is that the coding interview didn’t need AI to discredit it. The receipts have existed for years.

It measures anxiety, not ability. A randomized controlled trial from NC State and Microsoft found candidates solving problems in the traditional watched, think-aloud format performed roughly half as well as candidates solving identical problems alone — same skills, different stress [6][7]. The researchers noted the format is a near-perfect replica of the Trier Social Stress Test, the procedure psychologists use when they need to deliberately induce stress in lab subjects [8]. In the study, every woman in the public condition failed; every woman in the private condition passed [7]. (Small sample, the authors caution — but a chilling design signal.)

Its results are close to random. Across tens of thousands of interviews on interviewing.io, only about 25% of candidates perform consistently interview-to-interview; strong candidates bomb over a third of the time [9][10]. Practice volume predicted FAANG interview success better than measured technical ability did [11]. A process this noisy isn’t an assessment — it’s a lottery weighted by free time.

Its inventor already recanted. Google compared tens of thousands of its own interview scores to subsequent job performance and found “zero relationship” [12]. Laszlo Bock, Google’s then-head of People Operations, publicly called the company’s famous brainteasers a complete waste of time [13]. The industry copied Google’s interview. It never copied Google’s retraction.

So the baseline, pre-AI: a test with no demonstrated predictive validity, coin-flip repeatability, and a built-in stress penalty. Everything since has made it worse.

AI broke the test — in both directions

Direction one: software passes it. LeetCode-style problems have known, indexed solutions, and modern AI tools generate optimal answers in seconds [14]. In 2025 a Columbia student built Interview Coder, an invisible overlay that solved interview problems in real time; he claimed offers from Amazon, Meta, and TikTok, posted video proof, and turned the tool into a startup (Cluely) reporting $3M+ in annualized revenue within months [15]. In an interviewing.io survey of Big Tech interviewers, 81% suspected candidates of AI use and 31% had definitively caught someone; Amazon hiring managers told CNBC a majority of recent junior-to-mid candidates showed evidence of cheating [15]. HackerRank found 71% of hiring leaders say AI has made technical skill genuinely harder to assess [15]. Gartner projects a quarter of candidate profiles will be entirely fake by 2028 [14].

Direction two: the honest are punished twice. The candidate who follows your no-AI rule absorbs the full stress test, hand-writes code nobody hand-writes anymore, and competes against silent software. Your interview now selects for either stress-inoculated puzzle memorizers or comfortable cheaters. Neither is the hire you wanted.

And the deepest cut: even a “clean” pass measures a depreciating asset. When AI writes the boilerplate — and in data science, the pandas wrangling, sklearn pipelines, SQL, and plotting code are exactly what AI does best [3] — the scarce human skill shifts to what AI can’t do: framing the right question, knowing whether the data can support the conclusion, catching the subtly wrong AI-generated analysis, judging whether a model is safe to ship. The coding interview measures the commodity and ignores the scarce skill. It is, in the most literal sense, a legacy-skill assessment.

The competitive cost: what screening out AI-first talent does to your P&L

Here’s where this stops being a candidate-experience complaint and becomes a business-performance argument.

The performance gap between AI-leveraged and AI-lagging companies is enormous and widening. PwC’s 2026 Global AI Jobs Barometer — built on over a billion job ads plus company financials across 27 countries — found that companies most exposed to AI posted 34% productivity growth since 2018 versus 24% for the least exposed, and the top 20% of AI-exposed firms achieved a staggering 163% labor productivity growth — a genuine “super-star” separation [16][17]. The 2025 edition found revenue per employee in AI-exposed industries grew roughly three times faster than in less-exposed ones, with productivity growth nearly quadrupling since GenAI’s arrival [18][19]. At the individual level, a Stanford/MIT study of 5,000+ support agents found an AI assistant lifted resolution rates 14% on average — and 34% for less-experienced workers [20].

The market is already pricing AI-fluent people accordingly. Jobs requiring AI skills carried a 62% wage premium in 2026, up from 57% the year before and 25% two years prior — one of the fastest-repricing skills in modern labor data — and AI-skill roles are growing roughly eight times faster than the overall job market [16][17]. PwC’s data also shows a two-track split: “professionalised” roles, where AI automates the routine so human judgment carries the value, are growing twice as fast with 42% faster wage growth than roles AI merely simplifies [17]. An AI-first data scientist — someone who orchestrates AI to multiply their analytical output — is the textbook professionalised role.

Now connect the dots to your hiring funnel.

1. Your test filters for the depreciating skill and against the appreciating one. A no-AI, memory-based algorithm gauntlet awards zero points for the ability to direct, verify, and correct AI — the skill carrying a 62% market premium and driving the 163% productivity tier [16][17]. Worse, the candidates most fluent in AI-augmented workflows are the most likely to find hand-coding-from-memory alien, because they stopped working that way years ago. The most productive engineers at Anthropic and Spotify — people shipping 10–30 pull requests a day through agent orchestration [2] — are precisely the profile a LeetCode round is most likely to reject or repel.

2. Your test advertises your operating model — and AI-first talent is reading the ad. Interview format is the loudest signal a company sends about how work actually happens inside. A ban on AI in your assessment tells an AI-first data scientist that your workflows are years behind — exactly as the Interview Coder founder argued: companies billing themselves as AI-first while banning AI in interviews are advertising a contradiction [21]. Meanwhile Meta has rolled out an AI-enabled coding interview replacing one of its standard rounds, and a growing cohort of companies explicitly permit AI in assessments, reframing the question as “can you direct AI, catch its mistakes, and ship something good?” [15]. In a market where AI-skilled candidates command a 62% premium and are the scarcest resource in the funnel, the companies testing the actual 2026 workflow get first pick. The companies re-imposing whiteboards get whoever’s left.

3. You’re compounding it with the old exclusions. Everything the format always filtered out — it still does. Candidates with performance anxiety, whose measured ability drops by half under observation [6]. Neurodivergent candidates, for whom think-aloud, high-unpredictability, socially surveilled coding is arguably the most hostile format tech could have designed [22][23] — a talent pool with disproportionate strengths in pattern recognition and deep analytical focus, facing unemployment and underemployment rates estimated as high as 85% among college-educated autistic adults [24][25], with researchers and advocates consistently locating the barrier at the interview itself [25]. Women, who in the RCT all failed publicly and all passed privately [7], and who abandon the pipeline 7x more often than men after one noisy bad result [26]. Caregivers and lower-income candidates, taxed by a prep grind that predicts nothing but free time [8][11]. Every one of these is a subtraction from the same scarce pool your competitors are paying premiums to access.

The arithmetic is brutal. The productivity dividend of the AI era is concentrating in a super-star tier of companies [17]. Membership in that tier runs through people who can amplify themselves with AI. And the coding interview — no predictive validity, coin-flip reliability, stress-test design, AI prohibition — is a machine for rejecting those people at the door while your competitors hire them at a premium. Companies that keep it aren’t maintaining a quality bar. They’re running an adverse-selection engine against themselves.

“But take-homes, then?” — better instrument, usually botched

The reflexive fix is the take-home project, and in principle it’s right: it’s a work sample (top of the validity charts [27]), it removes the audience-stress penalty [6], and its asynchronous, predictable format is documented to help neurodivergent candidates [23]. But as commonly practiced, it recreates the old inequities and adds new failure modes.

The time tax discriminates again. Unbounded take-homes explicitly disadvantage people with families, second jobs, or tight finances — and candidates juggling several processes get several simultaneous multi-hour assignments [28].

Adverse selection, measurably. Before Dropbox moved away from take-homes, about 20% of candidates simply never completed them — and completions skewed toward weaker candidates, because the strong ones had competing offers and no appetite for roughly 15 hours of speculative labor against a roughly 10% pass rate [28]. A 2025 CoderPad survey found 62% of companies admit their take-homes are too long and often refused outright [29].

AI one-shots naive versions. One hiring manager reported all 15 take-home submissions in their inbox were structurally identical — every candidate had used AI [14]. An unsupervised artifact, evaluated as an artifact, now measures nothing.

The design that survives the AI era: timebox hard (2–4 hours), pay for the candidate’s time, explicitly allow AI — and move the evaluation weight to a structured, adaptive debrief. Why this approach? What did you validate? Where does this analysis break? What did the AI get wrong, and how did you catch it? AI can generate the deliverable; it cannot yet fake the candidate’s understanding of it under specific follow-up [14][15]. That conversation tests the exact skill the market is repricing: judgment over generation.

What the winners’ loop looks like

None of this requires invention. Personnel psychology settled the question decades ago: Schmidt and Hunter’s meta-analysis of 85 years of selection research put structured interviews and work samples at the top of the validity hierarchy (r ≈ .51, combinations ≈ .63) [27], and the major 2022 Sackett re-analysis kept structured interviews at the very top — more than double the predictive power of unstructured formats, with less bias [30][31]. The bottom of the charts is what most loops implicitly weight: pedigree, GPA, years of experience [12][31].

The evidence-based, AI-era data science loop:

1. A realistic, timeboxed, paid, AI-permitted work sample — messy data, fuzzy business question, the candidate’s real toolkit — assessed through a structured debrief on reasoning, verification, and communication.
2. Structure everywhere: same questions, anchored rubrics, calibrated interviewers. Structure alone roughly doubles validity and cuts bias [27][31], and its predictability directly lowers the barrier for neurodivergent candidates [23].
3. Materials in advance. Problem, format, criteria — before the interview. Unknowns cause anxiety and anxiety suppresses true ability [8]. The job contains no surprise whiteboard ambushes; the interview shouldn’t either.
4. Format choice — take-home, live pairing, written analysis, or a deep-dive on the candidate’s own past project. Documented to convert capable excluded candidates into hires [23].
5. Multiple data points, never one. Single-interview outcomes are near-arbitrary; aggregate signal is what’s reliable [10].
6. Test AI fluency directly. If your data scientists will spend their days directing and auditing AI output — and at the current trajectory, they will [1][2][3][4] — then AI-fluency-with-judgment is the core technical competency. Banning it from the interview is like banning calculators from an accounting interview in 1995, except the calculator now does 30% of the job [3][4].

The bottom line

Every previous critique of the coding interview was about fairness and validity, and companies could shrug it off as someone else’s problem. This one is about competitive survival, and it lands on the P&L.

The test never predicted performance — its own inventor published that [12]. Its results flip like a coin for the same candidate [9]. Its design is a clinical stress procedure that halves measured ability [6][8] and falls hardest on neurodivergent candidates, women, and the time-poor [7][22][26]. Software now passes it invisibly [15]. And the skill it isolates — hand-writing code from memory — is the single most automated activity in the profession, abandoned first by the industry’s most productive people [1][2].

Meanwhile, the productivity returns of this era are concentrating in a super-star tier of AI-leveraged companies [17], the market is paying a 62% premium for the people who create that leverage [16], and those people are choosing employers whose hiring process reflects how work actually happens now [15][21].

The companies that keep the coding interview will keep passing exactly the candidates AI just made replaceable, and keep failing exactly the candidates AI just made priceless. That’s not a hiring-process quirk. That’s how you end up on the wrong side of a 163%-to-24% productivity gap — one rejected AI-first data scientist at a time.

References

[1] Fortune, “The head of Claude Code hasn’t ‘written a line of code by hand’ in 8 months” (June 2026). https://fortune.com/2026/06/11/anthropic-claude-boris-cherny-doesnt-write-code-by-hand-anymore/

[2] Chiang, E., “The Man Who Built Claude Code Was Changed By It” (Feb 2026; summarizing Boris Cherny on Lenny’s Podcast, incl. Anthropic output figures, Spotify report, SemiAnalysis commit estimate). https://www.ernestchiang.com/en/posts/2026/the-man-who-built-claude-code-was-changed-by-it/

[3] TechCrunch, “Microsoft CEO says up to 30% of the company’s code was written by AI” (April 2025). https://techcrunch.com/2025/04/29/microsoft-ceo-says-up-to-30-of-the-companys-code-was-written-by-ai/

[4] Entrepreneur, “AI Is Already Writing About 30% of Code at Microsoft and Google” (2025; incl. Kevin Scott and Dario Amodei predictions). https://www.entrepreneur.com/business-news/ai-is-taking-over-coding-at-microsoft-google-and-meta/490896

[5] Saifi, S., “AI Writes 30% of Microsoft’s Code and 25% of Google’s” (Medium; measurement caveats). https://medium.com/@sohail_saifi/ai-writes-30-of-microsofts-code-and-25-of-google-s-6909f6e0b406

[6] NC State University News, “Tech Sector Job Interviews Assess Anxiety, Not Software Skills” (2020). https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/

[7] Behroozi, M., Shirolkar, S., Barik, T., & Parnin, C., “Does Stress Impact Technical Interview Performance?” ESEC/FSE 2020. http://www.chrisparnin.me/pdf/stress_FSE_20.pdf

[8] The Register, “Whiteboard coding interviews are ‘anti-women psychological stress examinations’” (2020). https://www.theregister.com/2020/07/15/it_hiring_whiteboard

[9] interviewing.io, “You can’t fix diversity in tech without fixing the technical interview.” https://interviewing.io/blog/you-cant-fix-diversity-in-tech-without-fixing-the-technical-interview

[10] interviewing.io, “After a lot more data, technical interview performance really is kind of arbitrary.” https://interviewing.io/blog/after-a-lot-more-data-technical-interview-performance-really-is-kind-of-arbitrary

[11] interviewing.io, “The technical interview practice gap.” https://interviewing.io/blog/technical-interview-practice-gap

[12] Bryant, A. / ERE, “Google’s Weird Interview Questions: ‘A Complete Waste of Time’” (2013, reporting Laszlo Bock’s NYT interview). https://www.ere.net/articles/googles-weird-interview-questions-a-complete-waste-of-time

[13] Quartz, “Google admits those infamous brainteasers were completely useless for hiring” (2013). https://qz.com/96206/google-admits-those-infamous-brainteasers-were-completely-useless-for-hiring

[14] Fabric, “Why LeetCode Interviews Are Now Vulnerable to Cheating” (2025). https://fabrichq.ai/blogs/why-leetcode-interviews-are-now-vulnerable-to-cheating-and-how-to-prevent-it

[15] SpaceComplexity, “Will AI Replace LeetCode Interviews?” (2025/26; interviewing.io interviewer survey, Interview Coder/Cluely timeline, HackerRank finding, Meta’s AI-enabled interview, Google onsite pilot). https://spacecomplexity.ai/blog/will-ai-replace-coding-interviews

[16] PwC, “2026 Global AI Jobs Barometer.” https://www.pwc.com/gx/en/services/ai/ai-jobs-barometer.html

[17] PwC press release, “AI reshapes global labour market into two distinct paths” (June 2026; 163% super-star productivity figure, 62% wage premium, 8x job growth). https://www.pwc.com/gx/en/news-room/press-releases/2026/pwc-2026-ai-jobs-barometer.html

[18] PwC US, “What AI means for your workforce strategy: 5 takeaways from the 2025 AI Jobs Barometer.” https://www.pwc.com/us/en/tech-effect/ai-analytics/ai-jobs-barometer.html

[19] PwC press release, “AI linked to a fourfold increase in productivity growth and 56% wage premium” (June 2025). https://www.pwc.com/gx/en/news-room/press-releases/2025/ai-linked-to-a-fourfold-increase-in-productivity-growth.html

[20] Open Data Science, “The AI Pay Divide” (summarizing the Stanford/MIT customer-support study and cross-source wage-premium estimates). https://opendatascience.com/the-ai-pay-divide-why-ai-skills-now-command-a-56-wage-premium-according-to-pwc/

[21] CNBC, “How Google is responding to AI cheating in coder interviews” (2025; incl. Interview Coder founder’s argument re AI-first companies). https://www.cnbc.com/2025/03/09/google-ai-interview-coder-cheat.html

[22] Glassdoor, “How to Conduct Interviews with Neurodiverse Candidates.” https://www.glassdoor.com/blog/how-to-conduct-interviews-with-neurodiverse-candidates/

[23] Chang, H.-Y., Saleh, M. C., Bruyère, S. M., & Vogus, T. J., “Making the employment interview work for a neurodiverse workforce,” Journal of Vocational Rehabilitation (2023). https://journals.sagepub.com/doi/10.3233/JVR-230031

[24] MyDisabilityJobs, “Autism and Employment Statistics” (2024). https://mydisabilityjobs.com/statistics/autism-employment/

[25] Organization for Autism Research, “NDEAM Highlights Barriers and Progress to Employment” (2024). https://researchautism.org/oaracle-newsletter/ndeam-highlights-barriers-and-progress-to-employment/

[26] interviewing.io, “Announcing the interviewing.io Technical Interview Practice Fellowship.” https://interviewing.io/blog/announcing-the-interviewing-io-technical-interview-practice-fellowship

[27] Schmidt, F. L., & Hunter, J. E., “The Validity and Utility of Selection Methods in Personnel Psychology,” Psychological Bulletin (1998); summary. https://firstpersonnel.org/wp-content/uploads/2013/10/Summary-Schmidt-Hunter-1998.pdf

[28] Holloway Guide to Technical Recruiting and Hiring, “Take-homes” (incl. the Dropbox completion/pass-through data). https://www.holloway.com/g/technical-recruiting-hiring/sections/take-homes

[29] dev.to / devtips, “AI just killed the coding interview” (2025; CoderPad survey and take-home length reports). https://dev.to/dev_tips/ai-just-killed-the-coding-interview-why-leetcode-wont-get-you-hired-anymore-1fd

[30] Criteria Corp, “Understanding the Latest Scientific Findings on Employee Selection” (on Sackett et al., 2022). https://www.criteriacorp.com/blog/updates-employee-selection-science

[31] Cogn-IQ, “Best Predictors of Job Performance: Schmidt-Hunter Hierarchy and the Sackett 2022 Update.” https://www.cogn-iq.org/blog/what-predicts-job-performance/

Brian Curry is a Kansas City–based cognitive systems architect and applied AI researcher working at the intersection of AI architecture, knowledge engineering, and intelligent-system design. He is the founder of Vector1 Research, an applied AI lab publishing on agent systems, AI evaluation, cognitive architecture, and the macroeconomics of AI-driven transformation. He has led applied AI and data initiatives across major enterprises including Koch Industries, Tractor Supply, Vail Resorts, Hallmark, Garmin, AT&T, and McClatchy, and founded KC AI Lab, a 1,500-member AI/ML community. His open-source work includes Papilon (causal inference and counterfactual simulation), PyCausalSim (causal discovery via agent-based simulation), MeaningFlow (semantic modeling and ontology), and Memory-Node Encapsulation (episodic memory for agentic systems). Contact: brian at vector1.ai · LinkedIn