Vector 1
Organizations & Work

The Caveman Ritual: Why Coding Interviews Cannot Hire the Polymaths the AI Era Demands

Brian J. Curry
Brian J. Curry
October 12, 2026 · 27 min read

Companies still asking data scientists to scribble hieroglyphics into a locked-down coding pad, webcam on and AI switched off, are not testing for the job. They are testing for the hiring manager’s past. Here is what the ritual actually measures, what it costs, and how to hire polymaths instead.

Brian Curry | Vector1 Research

In this article

  • Why the coding interview is an archaic ritual that the industry’s own data rejected more than a decade ago
  • What proctored coding interviews really measure, and why a candidate performs half as well with an audience
  • The LeetCode monkey: the candidate the ritual manufactures, and why AI made the species obsolete
  • How AI broke the ritual from both directions, and why the biggest companies run it anyway
  • Companies to avoid: twenty-two named employers that still run the ritual, Wall Street and the trading firms included, from their own guides and candidate reports
  • The math: why a strong candidate survives a five-round loop only 28.9% of the time
  • The gatekeepers: code monkeys grading LeetCode monkeys, and the psychology that keeps the ritual alive
  • The natural-conditions method for hiring polymaths, with techniques and a weighted rubric

The Premise

Picture the scene. A candidate with twenty years of experience across four industries, a person who has connected pricing to supply chains, rebuilt marketing measurement from first principles and taught an organization to see its own feedback loops, sits in front of a browser tab. The coding platform has forced full-screen mode, disabled paste, switched on the webcam and banned every AI tool. A hiring manager watches every keystroke in a shared editor. The task: reverse a linked list, from memory, in 45 minutes, while narrating every thought aloud.

The candidate is being asked to scribble hieroglyphics on a cave wall. The wall used to be a whiteboard; today it is a locked-down coding pad, and the change made the ritual stricter, not smarter. Nothing about the exercise resembles the job. No data, no business, no tools, no AI, no time to think, and no question worth answering. It is a ritual, and like most rituals, it survives because the people performing it have stopped asking what it is for.

This is the third article in the Data Science Is Jazz series. The first, “Data Science Is Jazz, and Your Engineers Can’t Play It,” argued that pure technicians, the code monkeys hired because they could pass a coding test, deliver the most average outcomes in data work. The second, “The End of the Vertical Data Scientist,” named the two kinds of practitioner: the monomath, with expertise in one lane, and the polymath, with decades of experience, a command of language and an associative mind that invents across domains. AI is replacing the first and amplifying the second.

This article is about the gate between them. Companies, hiring managers and recruiters still screen data scientists with a ritual designed for a different job in a different decade. It measures the skills AI made free and filters out the skills AI made scarce. Many of the managers enforcing it built their own careers as code monkeys, and they are now grading candidates against their own past instead of the job’s future. The candidates who pass are, more and more, LeetCode monkeys: people who drilled hundreds of puzzle problems until the answers became reflex.

The evidence against the ritual is not new, not subtle and not contested by the people who invented it. What follows is that evidence, the math of what the ritual actually selects, the psychology of why it persists, and a complete method for replacing it.

The Ritual and Its Retraction

The industry copied the interview and ignored the retraction

The puzzle interview, first on whiteboards and now on proctored coding platforms, spread through the technology industry largely by imitation of the companies that made it famous, Google above all. Then Google looked at its own data.

In a 2013 interview with The New York Times, Laszlo Bock, then Google’s senior vice president of people operations, said the company’s data showed its famous brainteasers did not predict job performance, and that it had found a “zero relationship” between how interviewers scored candidates and how those candidates later performed. Google moved toward structured interviews because that is what its evidence supported.

The industry did not follow. It had already copied the interview, built a training economy around it and promoted a generation of managers who passed it. The retraction never traveled as far as the ritual.

The biggest companies know, and run it anyway

In 2025, interviewing.io surveyed 67 interviewers, 52 of them at FAANG companies. Of the FAANG interviewers, 81% suspected candidates of using AI to cheat, and about a third said they had caught someone doing it. Only 11% said their company used cheating-detection software.

The most important number in the survey was zero. Not one of the 52 FAANG interviewers said their company had moved away from algorithmic questions. Some changed which puzzles they ask. None changed the ritual.

The most valuable technology companies on earth have a test they believe is compromised, that their own former people chief said predicts nothing, and that measures skills AI now performs in seconds. They are still giving it. Every company that copies their interview process copies that decision too.

The cave wall moved into the browser

Most coding interviews no longer happen at a whiteboard. They happen on platforms such as HackerRank, CodeSignal and CoderPad, and the platforms have turned the ritual into surveillance.

HackerRank’s Proctor Mode, according to its own documentation, disables copy and paste, flags every attempt to switch tabs or leave full-screen mode, captures a screenshot of the candidate’s computer every 15 seconds and runs a model over those screenshots to detect AI coding assistants and invisible overlay apps. An AI watches the webcam for a second face or a phone, and a gaze feature tracks whether the candidate’s eyes leave the screen. CodeSignal’s proctored assessments, by its own support guide, require the candidate to share camera, microphone and screen for the whole session and to show a government photo ID. CoderPad alerts the interviewer whenever a candidate pastes code or leaves the editor, and records a replayable timeline of every keystroke.

Read that list as a job description and it describes no job that exists. No analyst works with the clipboard disabled, the web forbidden, a camera tracking their gaze and every AI tool banned. The platforms did not modernize the cave wall. They put a guard in front of it.

The tools to do otherwise already exist. CoderPad’s own guidance tells interviewers they can enable an AI assistant for candidates so that every use of AI is visible inside the platform. Companies that switch AI off are not stuck with an old instrument. They are choosing to test for a world without AI because that is the world in which their managers were hired.

The LeetCode monkey

The ritual has produced its own species. LeetCode, the largest of the practice sites, hosts thousands of puzzle problems, and an entire training economy of courses, mock interviews and memorized pattern lists sits on top of it. The candidate it produces is the LeetCode monkey: someone who has drilled hundreds of problems until linked lists, sliding windows and dynamic programming tables come back on cue, and who has spent those months practicing the test instead of the work.

The LeetCode monkey is the code monkey’s apprentice, and the ritual rewards the apprentice for exactly what the master was rewarded for: fast recall of known solutions to known problems. That is the skill every AI coding assistant now delivers in seconds. A company that hires LeetCode monkeys in 2026 is paying a premium for the one capability it can already rent for the price of a subscription, and filtering out the framing, range and invention it cannot rent at any price.

Companies to Avoid for Polymaths, Advanced Data Scientists and Systems Thinkers

A hiring process is a signal. A company that screens data scientists with timed, closed-book coding has told every candidate what it values and how it thinks. For polymaths, advanced data scientists and systems thinkers, the companies below run some version of the ritual, documented either in their own published guidance or in candidate reports of their data scientist interviews.

Two caveats keep this list fair. Processes vary by team, level and year, and companies change them. Everything below reflects what each company published or what candidates reported, as of October 2026. A company that retires the ritual deserves to come off the list, and candidates should always ask a recruiter to describe every round before agreeing to it.

In their own words

Meta. Meta’s own candidate guide describes an initial technical screen of two coding problems in 35 minutes, followed by a full loop that includes coding rounds on algorithms and data structures, done on a whiteboard when the interview is in person. The guide advises candidates to practice writing code in a plain text editor without syntax highlighting or auto-completion. In other words, candidates are told to rehearse working without the tools every engineer uses every day.

Amazon. Amazon’s own 2024 guidance for software development engineer candidates says they “will be asked to write syntactically correct code” on topics that include data structures and algorithms, and advises practicing coding without an IDE. When a candidate said he had used an AI overlay to pass its coding interviews, Amazon declined to comment on the case, noting only that candidates must agree not to use unauthorized tools. The rule bans the tools. It does not question the test.

Google. The company whose own people chief said its interview scores predicted nothing is now bringing part of the process back into the room. In a 2025 conversation on the Lex Fridman Podcast, CEO Sundar Pichai said it was worth having “some fraction of the interviews being in person,” and reports described the change as a way to confirm candidates’ fundamentals as AI-assisted cheating rose. Faced with a test that no longer works, the response was to administer it under closer supervision.

Microsoft. Microsoft’s own careers guidance says in-person technical interviews will likely involve coding on a whiteboard, while phone interviews use a shared code editor so interviewers can watch candidates work in real time. That guidance dates to 2016, but candidate reports of Microsoft’s data scientist interviews describe the same pattern today: a timed HackerRank assessment of 30 to 60 minutes and LeetCode-style coding in the loop.

Bloomberg. Bloomberg’s own interview guidance says candidates are assessed on coding, data structures and algorithms, and may code on a laptop, a whiteboard or paper.

Wall Street and the trading firms

Finance deserves its own list. These firms hire quantitative researchers and data scientists in large numbers, pay among the highest salaries in the field and present themselves as the home of the sharpest minds. Their gates are among the most rigid.

Jane Street. Jane Street says in its own guidance that it avoids “algorithm bingo” and puzzles with clever tricks, and it deserves credit for open-ended problems. But the same guidance says its interviews are “mostly, if not exclusively, focused on collaborating on a series of coding problems,” over Zoom and then in person, in a real programming language rather than pseudocode. A better puzzle performed live under observation is still the ritual. Candidate reports for its quantitative and data roles add rounds of probability brainteasers, including one five-round onsite day that ended in brainteasers.

Citadel Securities. Citadel Securities’ own description of its quantitative research process says the first round, a 45- to 60-minute remote interview, will “focus on programming skills, ability, data structures/algorithms and problem-solving,” followed by an onsite of three to five 60-minute interviews.

Goldman Sachs. Goldman’s own guide to its HackerRank assessment says every candidate completes a 30-minute Software Fundamentals challenge of data structures and algorithms problems.

JPMorgan Chase. JPMorgan’s own posting for its 2026 Data Science Analyst Program lists “a coding challenge powered by HackerRank and a recorded video interview powered by HireVue” and states that both are required. A data science program screens its candidates with a timed coding test and a camera.

Two Sigma. Candidate reports compiled by Interview Query describe a data scientist process that opens with a three-question HackerRank assessment and includes a separate one-hour coding and algorithms interview.

Hudson River Trading. Candidate reports compiled by QuantVault describe an online assessment on HackerRank or CodeSignal of about 150 minutes for algorithm developers, which the site calls the firm’s biggest cut, followed by a programming screen on CoderPad and a full onsite day of harder programming rounds.

In candidates’ words: data scientist interviews

Candidate-reported interview guides compiled by Interview Query describe live or timed coding in the data scientist interviews at the following technology and consumer companies.

Google. A 45-minute graph problem rated LeetCode medium to hard, and SQL written in a document-style editor rather than run.

Meta. A SQL screen of roughly 45 minutes in which one candidate reported that the editor did not run queries, so the SQL had to be written blind.

Amazon. At least two coding challenges in a shared editor while interviewers watch, then five back-to-back 45-minute onsite interviews, with whiteboard coding common.

Apple. Algorithmic problems including the median of two sorted arrays and a k-means implementation written from scratch.

Uber. A second screen one candidate described as “a more interactive coding question that felt LeetCode-like.”

Airbnb. A 30-minute screen of SQL and Python problems, followed by a live coding round in the onsite loop.

LinkedIn. A live phone screen focused on SQL and data processing.

Pinterest. A 60-minute screen with two SQL, two Python and two statistics questions. Engineering candidates also report two or three medium-to-hard onsite coding rounds where data structures and algorithms carry the most weight, and one reported solving every problem optimally without receiving an offer.

Spotify. Live Python and SQL coding with reported time pressure, including FizzBuzz and a problem similar to Three Sum.

DoorDash. A timed 30-minute SQL assessment of three to four problems.

Lyft. A 90-minute phone screen of LeetCode and SQL problems at medium difficulty, and an onsite technical round closer to LeetCode Hard.

Capital One. A LeetCode-style Python problem on arrays, paired with a business case.

Instacart. A one-hour live coding round on CodeSignal at medium LeetCode difficulty.

TikTok. A timed online assessment of SQL and Python tasks, then live SQL or Python exercises in the onsite loop.

The pattern

That is twenty-two companies, many of them among the most admired employers in technology, finance and consumer software. Several run thoughtful conversations elsewhere in their loops. But each still asks data scientists or quantitative researchers to perform some version of a coding ritual that the evidence says does not predict the job.

Jane Street is the most instructive name on the list. It is a firm thoughtful enough to reject trick puzzles in writing, and it still built its process around live coding under observation. That is how deep the ritual runs: even the companies that see its flaws redesign the puzzle instead of retiring the performance. When a company insists on the ritual in any form, a polymath should read it as information about how that company thinks, and decide accordingly.

Hieroglyphics on a Cave Wall

It measures anxiety, not ability

In 2020, researchers at North Carolina State University and Microsoft ran a randomized controlled trial on the whiteboard interview itself. Mahnaz Behroozi, Chris Parnin and colleagues gave 48 computer science students the same technical problems in one of two settings: a traditional whiteboard interview with an interviewer watching, or a private room with no observer and no requirement to narrate.

The results, presented at the ACM’s software engineering conference ESEC/FSE, were stark. Participants interviewed in the traditional setting performed about half as well as those who solved the problems in private. Every woman in the public setting failed; every woman in the private setting passed, though the authors cautioned that the sample was too small for firm conclusions on that point. Parnin noted that many well-qualified candidates are eliminated simply because they are not used to whiteboarding in front of an audience.

The whiteboard does not measure whether someone can do the work. It measures whether someone can perform under a specific, artificial kind of observation that never occurs in the job. The proctored coding pad does not remove that observation; it multiplies it. The audience is now an interviewer watching keystrokes, a webcam, a screenshot every 15 seconds and a model scanning for forbidden tools. The study found that one observer was enough to cut performance in half.

It is not even consistent with itself

A test that measures the wrong thing might at least measure it reliably. The coding interview does not. An interviewing.io analysis of 299 interviews found that only about 25% of candidates performed consistently from one interview to the next. Strong candidates, averaging around 3 on a 4-point scale, failed a single interview up to 22% of the time.

If a predictive model flipped its answer that often on the same input, no data scientist would ship it. Companies ship it on people every day.

It is not on the leaderboard

The most rigorous recent review of hiring methods, by Paul Sackett, Charlene Zhang, Christopher Berry and Filip Lievens in the Journal of Applied Psychology (2022), ranked structured interviews first, with a mean validity of 0.42. Work samples came in at 0.33 and general cognitive ability at 0.31. The algorithm puzzle performed under observation is not a structured interview and not a realistic work sample. It is a ritual that borrows the costume of both.

AI Broke the Ritual From Both Directions

Software passes the test

Columbia student Chungin “Roy” Lee built Interview Coder, an invisible overlay that solved coding interview problems in real time, and said he used it to land an Amazon internship. Lee says Columbia suspended him. In April 2025 his company, Cluely, announced $5.3 million in seed funding. A business now exists whose product is passing the ritual.

The fraud is not confined to one tool. Gartner reported in 2025 that 6% of candidates in a survey of 3,000 admitted to interview fraud, and predicted that by 2028, one in four candidate profiles worldwide will be fake.

The honest are punished twice

A candidate who follows the no-AI rule absorbs the full stress of the proctored pad and competes against candidates with silent software. The ritual now selects for two kinds of people: LeetCode monkeys who have drilled puzzles until recall is automatic, and those willing to cheat. Neither is the polymath the job needs.

Even a clean pass measures the wrong thing

Suppose a candidate passes honestly. What has been certified? The ability to recall and produce code for a known problem under time pressure, which is precisely the work AI now does better, faster and cheaper than any human. A clean pass on the coding interview is a certificate of competence in the most commoditized skill in the profession. It is a legacy-skill assessment, and the legacy is ending.

The Math of the Ritual

A strong candidate rarely survives the loop

Most coding loops are not one interview. They are several, and in many companies one bad round sinks the candidate. If a strong candidate fails any single round with probability p, and the rounds are independent, the chance of surviving all k rounds is:

Using interviewing.io’s upper estimate of p = 0.22 for strong candidates, a strong candidate survives one round 78.0% of the time, three rounds 47.5%, four rounds 37.0% and five rounds just 28.9%. Under these conditions, a five-round loop with a single-round veto rejects more than seven in ten strong candidates. The rounds are not perfectly independent in practice, but the direction is unmistakable: stacking unreliable rounds with a veto compounds the noise.

What the loop selects for: a proposed model

The second question is what the loop rewards when it does pass someone. The following proposed model makes the assumptions explicit so readers can change them.

Each candidate has two independent traits. Execution, E, is syntax, recall and speed on known problems, the work AI now supplies. Polymath range, P, is framing, abstraction and cross-domain invention. Job performance mixes them, with w as the share of the job that is execution:

“Drill” is puzzle practice: it raises loop scores and nothing else. The work sample is the natural-conditions method described below: multi-day, real tools, AI allowed. Its noise is set high on purpose, so that for execution-heavy jobs its validity lands near the empirical 0.33 from Sackett and colleagues.

What does the coding loop actually select? A proposed model.

Two independent traits per candidate, standardized:
    E = execution: syntax, recall, speed on known problems (what AI now supplies)
    P = polymath range: framing, abstraction, cross-domain invention
Job performance mixes them, with w = the share of the job that is execution:
    Y = w*E + (1 - w)*P
Two ways to screen:
    coding loop  = E + 0.8*drill + noise(1.3)   timed, closed-book, under observation
    work sample  = 0.3*E + 0.9*P + noise(1.0)   multi-day, real tools, AI allowed
"drill" is puzzle practice: it raises loop scores and nothing else.
"""
import numpy as np

rng = np.random.default_rng(42)
N = 1_000_000
E, P, drill = rng.standard_normal((3, N))
coding_loop = E + 0.8 * drill + 1.3 * rng.standard_normal(N)
work_sample = 0.3 * E + 0.9 * P + 1.0 * rng.standard_normal(N)

def evaluate(w, top=0.10):
    Y = w * E + (1 - w) * P
    Y = (Y - Y.mean()) / Y.std()
    out = {}
    for name, score in [("coding loop", coding_loop), ("work sample", work_sample)]:
        hired = score >= np.quantile(score, 1 - top)
        out[name] = (np.corrcoef(score, Y)[0, 1], Y[hired].mean())
    return out

def strong_candidate_survival(p_fail=0.22, rounds=(1, 3, 4, 5)):
    return {k: (1 - p_fail) ** k for k in rounds}

if __name__ == "__main__":
    for k, s in strong_candidate_survival().items():
        print(f"strong candidate survives a {k}-round loop: {s:.1%}")
    print()
    grid = np.round(np.arange(0.0, 1.0001, 0.01), 2)
    gaps = [evaluate(w)["work sample"][1] - evaluate(w)["coding loop"][1] for w in grid]
    print(f"coding loop only wins when execution is more than {grid[np.argmax(np.array(gaps) < 0)]:.0%} of the job")
    print()
    for w in (0.2, 0.5, 0.9):
        r = evaluate(w)
        print(f"execution share w={w:.2f}: "
              f"coding loop validity {r['coding loop'][0]:.2f}, hires average {r['coding loop'][1]:+.2f} SD | "
              f"work sample validity {r['work sample'][0]:.2f}, hires average {r['work sample'][1]:+.2f} SD")
    top_poly = P >= np.quantile(P, 0.90)
    for name, score in [("coding loop", coding_loop), ("work sample", work_sample)]:
        hired = score >= np.quantile(score, 0.90)
        print(f"{name}: rejects {1 - hired[top_poly].mean():.0%} of the top-decile polymaths")

The results:

  • The coding loop only wins when execution is more than 67% of the job. Below that line, the natural-conditions work sample hires better people.
  • When execution is 20% of the job, the coding loop’s validity is 0.13 and its hires average +0.23 standard deviations of performance. The work sample’s validity is 0.69 and its hires average +1.21.
  • When execution is half the job, the coding loop reaches 0.39 validity and +0.68; the work sample reaches 0.61 and +1.08.
  • When execution is 90% of the job, the coding loop finally wins: 0.54 validity and +0.95, against 0.29 and +0.50.
  • The coding loop rejects 90% of top-decile polymaths, exactly the rate of drawing names from a hat, because nothing in it measures polymath range. The work sample rejects 57%.

Notice what that 90% scenario describes: a job that is almost entirely execution of known problems. That is the code monkey’s job, the one the LeetCode monkey trained for, and it is the job AI is absorbing. The coding interview is optimized for the role that is disappearing and blind to the role that is emerging.

The Gatekeepers: Code Monkeys Grading LeetCode Monkeys

Why the ritual survives its own evidence

If the evidence is this clear, why does the ritual persist? Because the people who run it passed it, and the psychology of passing a painful initiation is one of the best-documented effects in social science.

In 1959, Elliot Aronson and Judson Mills ran a now-classic experiment. Participants who endured a severe, embarrassing initiation to join a group rated that group as more appealing than participants who went through a mild one, even though the group itself was deliberately dull. Gerard and Mathewson replicated the effect in 1966 with a harsher initiation. The explanation is cognitive dissonance: people who suffer to get into a group justify the suffering by deciding the group, and the gate, must be valuable.

A generation of hiring managers suffered through the whiteboard to get their jobs, and now hands the next generation a proctored coding pad. Many of them made their careers as code monkeys: excellent at recall, syntax and speed on known problems, promoted for it, and now in charge of who gets hired next. When they defend the ritual, they are not defending a validated instrument. They are defending the meaning of their own initiation. “I had to do it” is not a validity argument. It is Aronson and Mills.

They hire their own reflection

The second force is similarity. Lauren Rivera of Northwestern studied hiring at elite professional service firms through 120 interviews with employers and observation of a hiring committee, published in the American Sociological Review in 2012. Evaluators favored candidates who resembled themselves in experiences and self-presentation, and concerns about this cultural match often outweighed concerns about productivity alone.

Put the two forces together and the outcome is predictable. A manager who rose by drilling puzzles designs an interview that rewards drilling puzzles, recognizes a fellow driller as a strong candidate, and reads a polymath’s non-linear career as a red flag. Code monkeys grade LeetCode monkeys, and the team that results is a mirror of the manager’s past. The interview does not test for the job. It tests for membership.

The recruiters’ filters are built for monomaths

The ritual starts before the interview. Recruiter screens and applicant tracking filters typically search for the vertical profile: a specific degree, a PhD, a FAANG name on the résumé, a minimum number of years in a named tool. Every one of those filters selects for depth in a single lane and against range. A candidate who spent twenty years connecting marketing, operations, finance and data, and who plays music or writes on the side, often never reaches a human at all.

Recruiters are not villains in this story; they are executing the specification they were given. But a keyword filter written by a monomath will find monomaths, at scale, forever.

What the Ritual Costs

The cost is not abstract. A process that rejects most strong candidates through noise, rejects nearly all polymaths by design and admits the LeetCode monkeys and the dishonest produces exactly the teams the first two articles described: convergent, high in correlation, optimizing every chord and missing every line. The first article showed that ten people who think alike produce a best idea worth less than three independent thinkers. The coding interview manufactures ten people who think alike.

It also sends the wrong signal to the people companies most need. The polymath with decades of context has options. Asked to scribble hieroglyphics into a locked-down coding pad with the webcam on, many will simply decline, and the company will never know who it lost.

The second article described the original “Giant Steps” session, where pianist Tommy Flanagan, handed the changes cold, struggled audibly while the prepared Coltrane flew. The coding interview puts every candidate in Flanagan’s seat and then concludes they cannot play. No serious band auditions musicians that way. Serious hiring should not either.

How to Hire Polymaths: The Natural-Conditions Method

The alternative is not to lower the bar. It is to measure the right thing under the conditions in which the work actually happens. Polymath work happens over days, with tools, context and AI. Assessment should too.

1. Rewrite the role before writing the interview

Most bad interviews start with a bad job description. A role defined around one use case, one tool stack or one department is a vertical role, and it will attract and select monomaths. Describe the problems the business cannot yet frame, the departments the role must connect and the decisions it should change.

2. Screen for range, not keywords

Instruct recruiters to read for range: careers that cross industries and functions, evidence of invention, a serious creative practice, strong writing. Remove the PhD requirement, the FAANG pedigree filter and the years-in-a-named-tool minimums from the screen. Each one is a proxy for depth in one lane, and depth in one lane is what AI now supplies.

3. Review real work

Ask for past analyses, writing, decks or projects, and talk through how one of them evolved: what the original question was, how it changed, what was borrowed from elsewhere and what was thrown away. A body of real work is the richest evidence a candidate can offer, and the coding interview ignores it completely.

4. Run a natural-conditions work sample

Give a messy, realistic problem with several data sources and no stated question. Let the candidate work over several days in their own environment, with every tool available and AI explicitly allowed, because the job allows it. Cap the expected effort at a few hours and pay for the candidate’s time. This is the instrument the model above calls the work sample, and work samples are already near the top of the validated methods.

5. Debrief like colleagues, with structure

Replace the proctored coding performance with a working conversation about the work sample. Ask every candidate the same core questions: why this approach, what was validated, where the analysis breaks and what would change the conclusion. Structure is what puts structured interviews at the top of Sackett’s ranking. The conversation can be collegial and still be consistent.

6. Use the five prompts

The second article introduced five prompts that separate monomaths from polymaths, and they belong in the debrief: the unasked use case, the borrowed method, the second-season question, the guide-tone question and the rest. Each asks the candidate to invent, connect or restrain, which no puzzle can test.

7. Test writing, because English is the new code

Have the candidate write the specification an AI agent would need to run the analysis. Score it for precision, completeness and the quality of the questions it anticipates. When the programming language is English, writing is the technical skill.

8. Check references across functions

Ask for references from at least two different functions. Polymaths are known on both sides of the gaps they bridge, and a finance partner’s account of a candidate is evidence no coding round can produce.

9. Remove the single-round veto

Aggregate the evidence instead of letting any one bad round end the process. The math above shows how a veto compounds noise. Decide on the full body of work, the debrief and the references, scored against a written rubric.

The Rubric

Score each dimension from 1 to 5 against written anchors, then weight:

  • Framing, 25%. Strong: restates the problem as a sharper question than the one given. Weak: answers the question exactly as asked.
  • Range, 20%. Strong: brings a method or analogy from another field and adapts it. Weak: reaches only for the standard library and the standard model.
  • Invention, 15%. Strong: finds use cases and solutions nobody requested. Weak: improves the original use and stops there.
  • Systems view, 15%. Strong: traces feedback loops, incentives and second-order effects across teams. Weak: treats the deliverable as the end of the work.
  • Language, 15%. Strong: writes a precise, testable specification for people and AI agents. Weak: vague where precision matters.
  • Restraint, 10%. Strong: knows when the answer is a simpler analysis or no model at all. Weak: builds whatever was asked for.

Code quality is not on the rubric. It is a floor, checked in the real work and the work sample, not a differentiator scored against judgment.

The Path Forward

Executives should ask one question of their data hiring process: does it measure the job as it exists after AI, or the job as it existed when the current hiring managers were hired? If the answer is the second, the company is spending its recruiting budget to filter out its future.

Hiring managers should retire the proctored coding pad, the HackerRank screen and the timed algorithm round for every data science, analytics and AI seat that frames problems. If the strongest defense of the current process is that everyone had to go through it, recognize that argument for what it is: an initiation rite defending itself.

Recruiters should rewrite their screens for range. Remove the pedigree and keyword filters, read for careers that cross boundaries, and treat a serious creative practice as a signal rather than noise.

Candidates who are polymaths should ask about the process before they enter it, offer their real work in place of a LeetCode round, and treat a company that insists on the ritual as having told them something important about how it thinks.

No band auditions a jazz musician by asking them to recite scales onto a cave wall while the bandleader watches. They hand over the instrument, play a tune together and listen. Stop hiring LeetCode monkeys. Stop asking polymaths to scribble hieroglyphics. Hand them the instrument and listen.

Sources

About the Author

Brian James Curry is the founder of Vector1 Research, an independent lab working on causal inference, marketing economics, cognitive AI architecture and the economics of intelligent automation. He has spent twenty-five years in data, search, digital media and AI, and his career follows the horizontal path this series describes.

He began in digital media and search leadership at McClatchy, AT&T and YP, where he led digital transformation. He moved into data science, working across paid media, SEO and site search, then spent more than four years as Lead Data and Applied Scientist at Koch Ag & Energy Solutions, part of Koch Industries, building predictive models, machine learning and experimentation programs across commodities, marketing and operations. He went on to Tractor Supply Company as Marketing Data Scientist and AI Developer, and to Vail Resorts as Advanced Analytics and Machine Learning. In 2016 he founded KC AI Lab, a Kansas City machine learning education community and incubator that grew to about 1,500 members.

His open-source and research work includes Papilon, a Python library for causal inference and complex systems simulation; PyCausalSim, a simulation-based media mix modeling framework; Memory-Node Encapsulation (MNE), a data structure for artificial episodic memory in cognitive AI; The Autonomy Economy, a macroeconomic model of AI-driven transformation; and The Geometry of Data, on the structure of high-dimensional data.

He holds a BS in economics from Kansas State University, where he competed as a college athlete. He is a musician, composer and painter, has trained in Brazilian jiu-jitsu, and is the father of two.

Vector1 Research: vector1.ai · Substack: brianjamescurry.substack.com · Medium: medium.com/@brian-curry-research · GitHub: github.com/Bodhi8 · LinkedIn: linkedin.com/in/brian-curry-research

This is Part III of the Data Science Is Jazz series.

Related research