The End of the Vertical Data Scientist

AI will eliminate the code monkeys, the PhDs, the FAANG stars and the memorization puppets. From 2026 forward, the data scientist who matters is the horizontal thinker: decades of experience, a command of literature and language, and a divergent, associative mind that abstracts across domains.

Brian Curry | Vector1 Research

In this article

  • Why data science from 2026 forward is jazz, not an assembly line
  • The two ways jazz musicians play the changes, and what they reveal about every data team
  • The monomath and the polymath: who AI replaces and who it amplifies
  • A worked example: how a week-by-week optimizer loses 27.4% of a season’s profit
  • Why AI plays every chord, and why the line still belongs to the polymath
  • Why the modern interview is broken, and five prompts that separate monomaths from polymaths

Data Science and Jazz

Every jazz musician eventually confronts the same question: when the chords change, what do you play? One answer is vertical. The player treats each chord as its own problem, spells it out, nails it, and moves to the next. The other answer is horizontal. The player hears the progression as a single arc and plays one continuous line across it, letting individual chords pass underneath.

George Russell, the theorist whose Lydian Chromatic Concept of Tonal Organization (1953) helped lay the groundwork for modal jazz, framed jazz history around exactly this divide. Coleman Hawkins played vertically, working systematically through each chord. Lester Young played horizontally, floating over the chords and letting forward motion carry the line. The greatest players, in Russell’s account, learned to do both.

Data science splits along the same fault line, and most organizations have never noticed. The vertical data scientist answers each request as its own chord: this ticket, this metric, this model, this dashboard, each one correct on its own terms. The horizontal data scientist hears the line running through all of them, and that line is where new ideas live: a method borrowed from another field, a use case nobody requested, a solution to a question no one had thought to ask.

This is the second article in the Data Science Is Jazz series. The first, “Data Science Is Jazz, and Your Engineers Can’t Play It,” argued that pure technicians, the code monkeys hired because they could pass a coding test, deliver the most average outcomes in data work. This piece names the habit of mind behind that failure. The code monkey is a vertical player. So is the PhD trained to dig one hole ever deeper, and so are the other credentialed specialists hired for depth in a single lane. Every chord, perfectly spelled. No line.

The stakes have changed. Generative AI is the most fluent vertical player ever built. Hand it any single chord, a prompt, a ticket, a function to write, and it plays a correct answer in seconds. What it cannot do on its own is hold the line across a hundred prompts, a fiscal year, or an organization. That is now the scarce skill in data science, and the code monkeys do not have it.

This article covers the jazz history behind the two approaches, what each looks like on a data team, a worked example with runnable code showing a vertical strategy losing 27.4% of a season’s profit to a horizontal one, and how to hear the difference when hiring.

The Reality

Stated plainly: data science from 2026 forward will be more like jazz and less like an assembly line.

For more than a decade, the discipline ran like an assembly line. Companies hired for stations: a SQL station, a Python station, a model-fitting station, a dashboard station. Tickets moved down the line, each person performed one step to specification, and the coding interview checked whether a candidate could work a station. Rote memorization, recall, syntax and boilerplate were the job.

AI now works every station on that line. It writes the query, recalls the formula, fits the standard model and builds the pipeline, faster and cheaper than any human. The assembly line is being automated, the way assembly lines always are.

What remains is the work no assembly line ever did: hearing the real problem behind the request, connecting ideas across fields, and inventing solutions and use cases nobody specified. That work is jazz, and its stars are horizontal thinkers with decades of experience.

The vertical personas built their careers on the assembly line: the code monkeys, the memorization puppets, the PhDs trained to dig one hole ever deeper, the FAANG stars fluent only against someone else’s specification. With AI handling rote memorization, recall, coding and the other low-level tasks, they will no longer be relevant to where the discipline is going.

Two Ways to Play the Changes

Vertical: spell every chord

Coleman Hawkins defined the vertical approach. His style was chord-based and harmonically dense, staying closely tied to the underlying harmony and outlining each chord as it arrived. His 1939 recording of “Body and Soul” became a landmark precisely because it navigated the harmony so thoroughly.

John Coltrane took vertical playing to its limit. In 1958 the critic Ira Gitler coined the phrase “sheets of sound” in his liner notes for Soultrane, describing Coltrane’s torrents of arpeggios and scale patterns, each one articulating the harmony at extraordinary speed. In May 1959 Coltrane recorded “Giant Steps,” a composition that cycles through three key centers, B major, G major and E flat major, moving by major thirds at a fast tempo.

The tune remains one of the most feared progressions in jazz. On the original recording, pianist Tommy Flanagan, given no rehearsal, struggled audibly through his solo. Coltrane, who had worked out his melodic patterns over the changes in advance, played through it flawlessly. Vertical mastery at that level is real, demanding and hard won.

Horizontal: play the line

Lester Young defined the opposite approach. His playing was relaxed, sleek and linear. Where Hawkins spelled each chord, Young drew long melodic lines that moved smoothly over the harmony, letting individual chords pass underneath without stopping to announce them.

The horizontal approach became a revolution in 1959 with Miles Davis’s Kind of Blue. Instead of dense chord changes demanding a new scale every bar or two, the band played over a few modes held for long stretches. The music asked a different question: not “which notes fit this chord?” but “where is this line going?” Russell’s work fed directly into that shift. In Russell’s own telling, his concept grew out of a question Miles Davis asked him: “I want to learn all the chords.”

Gunther Schuller, writing in The Jazz Review in 1958, gave the horizontal idea its sharpest analytical form. In “Sonny Rollins and the Challenge of Thematic Improvisation,” he showed how Rollins’s solo on “Blue 7” developed a single short motif across the entire performance, building thematic unity over time rather than treating each chord as a separate event. A great solo, Schuller argued, has architecture.

The masters play both

The most important point in Russell’s framework is that the best players do not choose. Coltrane, the supreme vertical virtuoso of “Giant Steps,” moved toward modal, horizontal playing in the years that followed. Russell singled out pianist Bill Evans as a player who fused both approaches, hearing every chord and the line through them at once.

The bridge between the two is a technique every jazz student learns: guide tones. In any chord, the third and the seventh carry its identity. In a ii-V-I progression in C major (D minor 7, G7, C major 7), those tones form two smooth lines. The third of D minor 7 is F; it holds as the seventh of G7, then falls a half step to E, the third of C major 7. The seventh of D minor 7 is C; it falls a half step to B, the third of G7, and holds as the seventh of C major 7.

Each chord is correctly defined, which is vertical accuracy. Yet the notes move by the smallest possible steps, forming a continuous melody, which is horizontal connection. The guide-tone line is the proof that the two approaches are not opposites. Mastery is hearing both at once.

The monomath and the polymath

Data science needs names for its two kinds of practitioner, and the words already exist. The vertical practitioner is a monomath: someone with expertise in a single field, the rare but exact antonym of polymath. The monomath’s depth is real, and it lives in one lane: one paradigm, one stack, one body of memorized answers.

The horizontal practitioner is a polymath: decades of experience across industries and disciplines, a command of literature and language, and an associative mind that carries ideas from one field into another. The difference between them is not intelligence or effort. It is range.

The Monomath

Every ticket, perfectly spelled

The monomath treats every request as an isolated chord. The marketing team asks for a churn model; the vertical player builds an excellent churn model. Finance asks for a revenue forecast; the vertical player builds an excellent forecast. Operations asks for a dashboard; the dashboard is clean, fast and accurate.

Each answer is correct. No one checks whether the churn model’s targeting quietly inflates the revenue forecast’s assumptions, or whether the dashboard’s headline metric rewards behavior that destroys value two quarters later. The monomath was never asked, and never thinks to ask.

This is the code monkey’s native mode. The coding interview trained it: a closed problem, a known optimal answer, a clock. The software career rewarded it: a ticket with a specification, a test suite that turns green. Business vocabulary even names the structure. Companies organize into “verticals,” and the monomath serves each one in isolation, faithfully reproducing the organization’s silos inside its analytics.

Four kinds of monomath

The code monkey is the purest monomath, but not the only one. Several common, well-credentialed backgrounds train the same habit of mind.

The PhD. Five or more years spent going deeper into one question within one paradigm. The dissertation is the ultimate vertical exercise: one chord, spelled more precisely than anyone has spelled it before. It produces real depth, and it trains the instinct to dig the same hole deeper rather than ask whether the hole is in the right place.

The five-year FAANG engineer. Fluent at enormous scale, but always against a specification written by someone else, inside one company’s systems. Every sprint is a chord, and the product manager holds the line.

The new graduate with an advanced degree. Trained on benchmark problems with a fixed dataset and a single metric to beat. A benchmark is a chord frozen in place.

The memorization puppet. A career built on recalling the right answer to a known question: certifications, frameworks, interview patterns. Recall is the most vertical skill there is, and the first one AI made free.

Each of these monomaths can spell any chord on demand. None of their training ever asked for the line.

Rightness instead of richness

Edward de Bono drew the same distinction for thinking itself. In Lateral Thinking (1970), he wrote that “rightness is what matters in vertical thinking,” and that vertical thinking “selects a pathway by excluding other pathways.” His most famous line describes the code monkey’s career exactly: “You cannot dig a hole in a different place by digging the same hole deeper.”

The monomath digs deeper. A better model on the same question. A faster pipeline for the same report. A higher score on the same metric. The work improves locally while the organization’s position stays where it was.

The metric trap

Vertical thinking has a signature failure, and economists named it decades ago. Charles Goodhart observed in 1975 that any statistical regularity tends to collapse once pressure is placed on it for control purposes. The anthropologist Marilyn Strathern later gave it the form most people know: “When a measure becomes a target, it ceases to be a good measure.”

A monomath handed a target optimizes it, chord by chord, quarter by quarter. Because each step is locally correct, nothing in the monomath’s process detects that the target has stopped measuring what matters. Detecting that requires hearing the line: how this quarter’s optimization changes the system that will produce next quarter’s data.

The Polymath

The polymath’s real product is invention: solutions nobody specified and use cases nobody requested. Vertical thinking reproduces known answers. Horizontal thinking finds new ones, because new answers almost always live in the connections between chords.

This is where divergent thinking, creativity and innovation actually come from. The first article in this series showed that divergent thinking, generating many genuinely different approaches to one problem, is the rarest input in data work. Horizontal thinking is its working method: the habit of moving across chords, fields and departments instead of drilling down into one.

Playing outside the chord

In jazz, horizontal freedom is what makes invention possible. A vertical player is bound to the notes each chord allows. A horizontal player can pass through tension, borrow from other keys and land somewhere the progression never pointed, as long as the line resolves. On Kind of Blue, freeing the soloists from rapid chord changes did not simplify the music. It gave them room to invent melodies a chord-by-chord approach would never have produced.

Data work follows the same rule. The monomath’s toolkit is a repertoire of known chords: memorized syntax, memorized formulas, the standard model for the standard question. Pecking away at syntax and regurgitating formulas can reproduce any solution that already exists. It cannot produce one that does not. De Bono drew the line in the same place: “Richness is what matters in lateral thinking.”

Novel solutions live in the connections

The polymath finds new solutions by carrying an idea from one line to another. Three patterns recur.

A churn problem heard as an epidemic. The first article in this series showed that the Bass model of product adoption and a standard outbreak model are the same equation. The practitioner who hears churn or adoption as contagion inherits a century of epidemiology: seeding, thresholds, superspreaders.

A pricing problem heard as a memory problem. The worked example below solves a promotion problem by borrowing the reference price from behavioral economics. The breakthrough came from psychology, not from a better discount optimizer.

A forecast heard as a game. When a prediction changes the behavior it predicts, as with fraud, competitor pricing or credit risk, the problem is game theory rather than time series. Treating it as a forecast produces a model that degrades the moment it is deployed.

None of these is a better answer to the question as asked. Each is a different question, which is where breakthroughs come from.

Use cases nobody asked for

The monomath waits for the ticket. The polymath finds the use case. Data collected for one purpose almost always describes something else as well. Support transcripts are product research. Delivery timestamps are a demand signal. Returns are quality data. Searches that return nothing are a map of unmet demand.

Spotting those use cases requires hearing lines that vertical thinking cannot see. Across time, every decision changes the data the next decision will see. Across departments, the system is connected even when the org chart is not. Across feedback loops, outputs become inputs. Each of these lines is a place where use cases hide.

These are the Bridges and Vacuum Nodes of the first article: the people who import ideas across boundaries and connect what the org chart separates. Their value is not a better chord. It is a melody nobody else heard.

The Promotion Calendar: A Worked Example

The difference between vertical and horizontal play can be measured. Consider one of the most common decisions in retail analytics: how deeply to discount each week.

Marketing research has established that customers form a reference price from the prices they have seen, and that they react more strongly to prices above that reference (losses) than to prices below it (gains). Kalyanaram and Winer summarized these as empirical generalizations in Marketing Science in 1995. A discount sells more today. It also lowers what customers expect to pay tomorrow.

The following proposed model captures that dynamic. Price p is relative to the regular price, so p = 1 is full price and p = 0.7 is 30% off. The reference price r starts at full price and updates every week:

demand = 100 · exp(−2·(p − 1) + 3·max(r − p, 0) − 4.5·max(p − r, 0))
profit = (p − 0.5) · demand
r_next = 0.7 · r + 0.3 · p

Losses weigh 4.5 against 3 for gains, reflecting loss aversion. A 30% discount from full price multiplies demand by about 4.48 in the first week.

The vertical plan plays each chord: every week, it chooses the discount that maximizes that week’s profit. The horizontal plan plays the line: dynamic programming values each week’s decision by its effect on every week that follows.

Vertical vs horizontal: a 52-week promotion calendar.

Each week a retailer picks a discount. Customers remember past prices in a
reference price r, and react to the gap between today's price p and r:
a price below r feels like a gain, a price above r feels like a loss, and
losses weigh more than gains (loss aversion).

    demand(p, r) = 100 * exp(-BETA*(p - 1) + GAIN*max(r - p, 0) - LOSS*max(p - r, 0))
    profit(p, r) = (p - COST) * demand(p, r)
    r_next       = ALPHA * r + (1 - ALPHA) * p

Vertical plan: pick the discount that maximizes this week's profit, every week.
Horizontal plan: dynamic programming across the whole season.
"""
import numpy as np

COST, BETA, GAIN, LOSS, ALPHA = 0.5, 2.0, 3.0, 4.5, 0.7
DISCOUNTS = np.array([0.0, 0.1, 0.2, 0.3])
WEEKS, PLANNING_WEEKS = 52, 104          # plan two seasons ahead, report one
GRID = np.round(np.arange(0.60, 1.0001, 0.001), 3)

def profit(p, r):
    ref = np.where(p <= r, GAIN * (r - p), -LOSS * (p - r))
    return (p - COST) * 100 * np.exp(-BETA * (p - 1.0) + ref)

def next_ref(p, r):
    return ALPHA * r + (1 - ALPHA) * p

def vertical(r=1.0):
    """Play each chord: maximize this week, ignore what it teaches customers."""
    weekly = []
    for _ in range(WEEKS):
        prices = 1 - DISCOUNTS
        p = prices[np.argmax(profit(prices, r))]
        weekly.append(profit(p, r))
        r = next_ref(p, r)
    return np.array(weekly), r

def horizontal(r=1.0):
    """Play the line: value every week by what it does to all the weeks after it."""
    value, policy = np.zeros(len(GRID)), []
    for _ in range(PLANNING_WEEKS):
        options = [profit(1 - d, GRID) + np.interp(next_ref(1 - d, GRID), GRID, value)
                   for d in DISCOUNTS]
        policy.append(np.argmax(options, axis=0))
        value = np.max(options, axis=0)
    policy.reverse()
    weekly = []
    for t in range(WEEKS):
        p = 1 - DISCOUNTS[policy[t][np.argmin(abs(GRID - r))]]
        weekly.append(profit(p, r))
        r = next_ref(p, r)
    return np.array(weekly), r

if __name__ == "__main__":
    v, v_ref = vertical()
    h, h_ref = horizontal()
    print(f"vertical:   season profit {v.sum():7.1f}, week 1 {v[0]:5.1f}, week 52 {v[-1]:5.1f}, "
          f"ending reference price {v_ref:.2f}")
    print(f"horizontal: season profit {h.sum():7.1f}, week 1 {h[0]:5.1f}, week 52 {h[-1]:5.1f}, "
          f"ending reference price {h_ref:.2f}")
    print(f"horizontal advantage: {100 * (h.sum() / v.sum() - 1):.1f}%")
    behind = np.argmax(np.cumsum(v) < np.cumsum(h)) + 1
    print(f"vertical plan falls behind for good in week {behind}")
    print(f"full price after customers are trained to 0.70: {profit(1.0, 0.70):.1f} per week")

The results are stark.

The vertical plan discounts 30% every one of the 52 weeks, because every week the local math says to. In week 1 that earns 89.6 against 50.0 at full price. For the first ten weeks the vertical plan leads on cumulative profit, and any weekly or monthly dashboard would show it winning.

Underneath, the reference price is falling toward 0.70. Each week’s discount earns a little less than the last, and from week 11 onward the vertical plan is permanently behind. By week 52 it earns 36.4 a week. Worse, it is trapped: with customers trained to expect 30% off, returning to full price would earn just 13.0 a week. Every chord was played correctly, and the line led into a corner.

The horizontal plan never discounts. It holds full price, keeps the reference price at 1.00 and earns 50.0 every week. Over the season it earns 2,600.0 against 2,041.6 for the vertical plan, a 27.4% advantage. This matches the formal result of Kopalle, Rao and Assunção in Marketing Science (1996): when consumers weigh losses more heavily than gains, a constant price is optimal. When gains dominate, the optimal policy becomes a cyclical high-low pattern. Which regime applies is itself a horizontal question about the market as a whole, not about any single week.

Note what the horizontal solution actually was. It was not a better discount. It was a different idea, imported from behavioral economics: the customer has a memory, and every price teaches it something. No week-by-week optimizer could have found that, however fast, because the idea was not in any single week’s chord.

In jazz terms, the horizontal plan’s best move was the rest. Knowing when not to play is the most horizontal skill there is, and no single-chord analysis can discover it.

Two features of this example generalize. First, the vertical plan looks better at the start, which is precisely when most organizations evaluate their analysts. Second, its failure is invisible inside each decision; it appears only in the trajectory. Code monkeys, optimizing ticket by ticket, will produce this pattern reliably, and the quarterly review will reward them for it until the trap closes.

AI Plays Every Chord. The Line Is Yours.

Generative AI is a vertical virtuoso. Give it a well-specified chord, write this query, fit this model, summarize this report, and it returns a fluent, often excellent answer in seconds. Like Coltrane over “Giant Steps,” it has effectively pre-learned patterns for every progression it has seen.

What it does not do on its own is carry intent across a hundred separate prompts, notice that this month’s answer contradicts last quarter’s, or recognize that a locally optimal recommendation is walking the business into a trap like the promotion calendar’s. The line across prompts, across time and across the organization belongs to whoever is directing the tool.

AI is also the most capable regurgitator ever built. Syntax, standard formulas, textbook models: everything the memorization puppet and the code monkey sell, a model retrieves instantly. Left to itself, it tends toward the most typical answer to the question as asked, the center of what it has seen rather than the edge. Novel solutions come from asking a different question, and choosing which question to ask is the polymath’s job.

That divides the profession cleanly. The monomath’s entire value was vertical: correct answers to well-specified questions. AI now supplies that at near-zero cost, so the code monkey and every other monomath are competing head-to-head with the machine on the machine’s best ground. The polymath uses AI to play every chord faster and spends the saved time on the line. In the terms of the first article, V ≈ A × C: AI multiplies the context the human brings, and horizontal context, the map of how the system connects over time, is exactly the context it multiplies.

The division of labor follows directly. Rote problems and routine tasks, the syntax, the standard formulas, the well-specified ticket, are not where value creators belong. That work should fall to AI, which does it faster, cheaper and without fatigue. An organization that keeps its most creative people pecking at syntax is spending its scarcest asset on its cheapest work. Horizontal operators belong on the problems no tool can frame: the novel solution, the borrowed method, the use case nobody asked for.

Earning the Horizontal

Horizontal play is not a shortcut around vertical skill. In jazz, a player who drifts over chords without knowing them is not playing horizontally; they are simply lost. Lester Young knew the changes cold. Coltrane mastered vertical playing to the limit before moving beyond it.

The same holds in data science. The polymath knows the statistics, the models and the code, and can spell any chord on demand. What separates them is that they stopped treating the chords as the music. That transition takes years, and it maps directly onto the experience argument from the first article.

The recent graduate has not yet mastered the chords, so horizontal play is out of reach. The code monkey mastered the chords and never left them; a career of tickets and coding tests rewarded vertical play and never asked for a line. The polymath, with 15 or more years across eras, industries and functions, has watched enough decisions play out over time to hear where a line is going before it gets there. Like Bill Evans in Russell’s account, they hear every chord and the line through them at once.

How to Hear the Difference When Hiring

The audition is broken

No jazz musician composes a masterpiece in a one-hour audition, without their own instrument, without a studio, without the chance to edit and without time to settle into the creative state the work demands. Yet that is exactly how most companies try to hire polymaths.

The original “Giant Steps” session shows what cold conditions do even to a master. Tommy Flanagan, one of the finest pianists of his generation, was handed the changes without rehearsal and struggled audibly through his solo. Coltrane, who had prepared his approach in advance, played it flawlessly. The difference that day was not talent. It was preparation and conditions. The standard technical interview puts every candidate in Flanagan’s seat, then concludes they cannot play.

Polymath work does not happen on a clock in a conference room. It happens over days: reading, sketching, letting an analogy surface from another field, drafting, discarding and returning. It depends on tools: the practitioner’s own environment, their notes, their libraries and AI. An interview that strips away time, tools and context strips away the exact conditions in which abstract, cross-domain thinking operates. What remains measures composure under artificial pressure, a different skill and largely a vertical one.

The modern interview is broken for roles with complex, abstract, cross-domain requirements, and the fix is to assess candidates the way they actually work.

  • Review real work. Study past analyses, writing and projects, including how the problem evolved, not only the final result.
  • Give real time and real tools. Set a paid problem over several days, in the candidate’s own environment, with every tool available, AI included.
  • Talk like colleagues. Replace the whiteboard performance with a working conversation about choices, dead ends and borrowed ideas.

Why the modern interview fails polymaths, and what should replace it, is the subject of the next article in this series.

Five prompts that separate them

Monomaths and polymaths sound identical on a single chord. A coding test is a single chord, which is why it cannot tell them apart. The difference appears when the problem is open and the conditions are real. Within those conditions, five prompts reveal which kind of player is in the room.

The unasked use case. Hand the candidate a dataset collected for one purpose, such as support transcripts or delivery logs, and ask what else it could be used for. Count the use cases that are genuinely new rather than variations on the original. The monomath improves the original use. The polymath finds three nobody requested.

The borrowed method. Describe a business problem and ask how another field has already solved something with the same structure. Listen for epidemiology, ecology, economics or music. Code monkeys reach for a library. Polymaths reach for an analogy.

The second-season question. Present a locally successful result, such as a promotion that lifted weekly revenue. Ask what it did to the following quarter. The monomath defends the lift; the polymath asks what customers learned.

The guide-tone question. Ask how a model’s output will become someone else’s input, and how that input will return as training data. Listen for whether the candidate can trace the loop without prompting.

The rest. Ask when the right move is not to build the model at all. Polymaths have answers ready. Code monkeys rarely understand the question.

The Path Forward

The organizations that win with AI will put vertical work where it now belongs, in the tools, and put the polymaths in charge of the line.

Executives should audit where their analytics organization optimizes locally: which metrics each team owns, which models feed which decisions, and where a locally successful number could be walking the business into a trap. Every department-level metric with no owner of its downstream effects is a chord with no line. Then move the rote work to AI and point the polymaths at invention, where their value compounds.

Hiring managers should stop auditioning polymaths under conditions no musician would accept. Replace the coding test with open problems worked in natural conditions, with real time, real tools and AI: a dataset with no stated question, a problem to solve with another field’s method, a question about second-order effects. Hire the people who find what nobody asked for, and stop hiring code monkeys and other monomaths for seats that need the line.

Practitioners should treat every deliverable as one chord in a longer progression. Before building, ask what the work will teach the system and what the next decision will inherit from it. Let AI spell the chords. Spend the reclaimed time on the line: the borrowed idea, the unasked use case, the question nobody thought to ask.

Hawkins and Young both made history, and the music needed both. But in an era when a machine can spell every chord in the book, the musicians who matter are the ones who know where the line is going.

AI will replace the monomaths. The polymaths will direct it.

Sources

Brian Curry writes on applied AI, analytics and the economics of data work at Vector1 Research. Read the first article in this series, “Data Science Is Jazz, and Your Engineers Can’t Play It,” and subscribe on Substack at brianjamescurry.substack.com.

Related research