Agentic AI Is Just Another Abstraction Layer: What Data Scientists Should Offboard, and What They Can’t

The same coding agent gives a data scientist back anywhere from minus 3.5 to plus 11.5 hours a week. The difference is the person directing it.

Every generation of programming has moved the work up a layer. Compilers, and then languages like Python, took programmers up from raw binary and assembly, and the layer below became infrastructure. Coding agents are doing the same thing to code. Models, agents and the systems around them are becoming infrastructure under a data scientist’s highest-level work: deciding what to build, how the pieces fit, and whether the answer is right.

This teaser covers the ideas. The full deep dive on Substack has the math, the Python and the playbook for running agents in production.

Prompts are the new programming language

If agents are the new infrastructure, then prompts, instructions and context are the programming language written on top of it. A brief or an AGENTS.md file is source code, and the agent plays the part the interpreter used to.

The difference from Python is what the language demands of the writer. It has almost no syntax and an unbounded vocabulary, so it can only express what the writer knows. A brief can name the grain of a table, the right definition of churn or the leak to avoid only if its author knows those things. Two people with the same model and the same data write different programs, because they think differently and have seen different things in their domain.

One thing is new. A compiler is deterministic. An agent is probabilistic, so this layer leaks far more than any compiler did, and the checks that catch the leaks are part of the infrastructure. The full article includes a check suite in which a single join quietly went from 1,000 rows to 1,468.

Agents are inert and unreliable

An agent does nothing until directed, starts every session with no memory of the business, and carries out a wrong plan with the same diligence as a right one. It is also unreliable in a way that compounds: R = p^n. At 95% per step, a 20-step run succeeds 35.8% of the time.

Checks change that more than better prompts do. Add a check that catches 90% of bad steps and allow two retries, and the same agent finishes 89.9% of runs. What remains is mostly silent error, the kind that ships a wrong number and nobody notices. The full article walks through the checkpoint model in Python.

The reviewer is a variable in the equation

Some catching is automated. The rest is you. In a proposed model, hours freed per week are F = H × a × (1 − r − q × e), where H is the offboardable work, r is review time and q is the silent error rate at your catch rate. Anaconda's 2022 survey puts data prep at 37.75% of a data scientist's time and model work at 26.44%, which works out to about 20.4 offboardable hours a week.

Hold the agent fixed and vary only the reviewer. With no real review, offboarding costs 3.5 hours a week. A reviewer new to the domain gets back 3.9. A reviewer with decades of cross-domain context gets back 11.5.

It is easy to misjudge this. In METR’s 2025 trial, experienced developers expected AI to speed them up 24%, believed afterward it had sped them up 20%, and were measured 19% slower. Measure the hours instead of trusting the feeling.

Offboard the execution, keep the judgment

Offboard the work you can check faster than you could do it: data prep, feature code, proof-of-concept models, tests and boilerplate. Keep the work that depends on knowing the business: domain synthesis, strategy, orchestration, direction and divergent thinking.

That last one needs protecting. In Doshi and Hauser’s 2024 study, stories written with one AI idea were 10.7% more similar to each other. Write your own hypotheses first. Use the agent to execute your strange idea, not to supply it.

What’s in the Full Article

• The full offboard-versus-keep table, and why building agents is still deep technical work
• Python for the reliability model, the offboarding equation and a data check suite
• The AGENTS.md template I use for analytics repos
• A promotion gate that decides which agents earn a production slot, with code
• A guarded coding-agent loop with budgets, permission tiers and a trace

For data scientists, analytics leaders and CTOs deciding how much work to hand to agents, and who should be directing them.

Read the full deep dive on Substack → HERE

Subscribe on Substack to get each piece as it publishes.

Brian Curry writes about artificial intelligence/super intelligence in Kansas City: agentic AI, applied machine learning, causal inference and forecasting at Vector1 Research.