Building LLM Apps and AI Agents Is Not Data Science. Stop Calling It That.

The AI hype cycle is creating a new generation of mislabeled roles — and it’s about to burn out an entire profession all over again.

I’ve watched this movie before.

A decade ago, companies decided that “Data Scientist” was the hot title, and they promptly used it to describe every job that touched a number. Data engineers, BI analysts, analytics engineers, ML engineers — all mashed into one impossibly broad role with one salary and the expectations of five people. The industry spent years sorting out the damage.

Now it’s happening again. Except this time the thing getting crammed under the data science umbrella isn’t dashboards and data pipelines. It’s large language models, autonomous agents, RAG systems, and generative AI products.

Open LinkedIn right now. Search for “Data Scientist” roles posted in the last month. Count how many of them ask for experience with LangChain, vector databases, prompt engineering, fine-tuning LLMs, and building agentic workflows — often on top of the usual statistics, machine learning, SQL, and communication requirements.

This is not a data science job. It’s a fundamentally different discipline wearing a borrowed name tag. And the consequences of pretending otherwise are about to get expensive.

Let’s Start With What Data Science Actually Is

This needs to be said plainly, because the term has been diluted so aggressively that people have forgotten.

Data science is the practice of extracting knowledge and insight from data using scientific methods. A data scientist works with structured problems: there’s a question, there’s data, you apply statistical reasoning and modeling to arrive at an answer, and you quantify how confident you are in that answer.

The daily work involves things like designing A/B tests with proper sample size calculations, building classification or regression models on tabular data, performing exploratory data analysis to identify patterns and anomalies, applying causal inference techniques to estimate the impact of a business decision, and communicating findings to stakeholders with appropriate uncertainty.

The intellectual core of the discipline is statistics. The tools are Python or R, SQL, scikit-learn, pandas. The outputs are insights, models with measurable performance metrics, and recommendations grounded in evidence.

This is valuable, rigorous, important work. And it has almost nothing to do with building an LLM-powered application.

Now Let’s Look at What LLM and Agentic AI Work Actually Involves

When a company says they want someone to “build AI products” using large language models or autonomous agents, here’s what the actual work looks like.

Prompt Engineering and System Design

The person designing prompts for an LLM-powered system is not fitting a model to data. They’re writing structured natural language instructions that steer a nondeterministic system toward desired behavior. This involves crafting system prompts, designing few-shot examples, implementing chain-of-thought reasoning patterns, testing edge cases in model behavior, and iterating on instruction sets that balance specificity with flexibility.

This is closer to writing a detailed specification for a system that interprets language than it is to anything in a statistics textbook. The skill set is part technical writing, part systems thinking, part understanding the quirks and failure modes of transformer architectures. A data scientist who has spent five years building gradient boosting models on customer transaction data has no particular reason to be good at this.

RAG Pipeline Architecture

Retrieval-augmented generation is one of the most common patterns in LLM applications, and building one properly is a genuine engineering challenge. The work includes chunking documents with strategies that preserve semantic coherence, selecting and fine-tuning embedding models, configuring vector databases with appropriate indexing and search parameters, implementing hybrid retrieval that combines semantic and keyword search, managing context window budgets, and building the retrieval-generation interface that determines what the model actually sees.

This is information retrieval engineering. The relevant background is in search systems, distributed databases, and NLP — not in experimental design or regression analysis.

Agent Design and Orchestration

Building an agentic system — one where an LLM makes decisions, calls tools, and executes multi-step workflows — is a software architecture problem. The work involves defining tool schemas and function interfaces, designing state machines that govern agent behavior, implementing planning and reasoning loops, building guardrails that prevent the agent from taking harmful or nonsensical actions, handling error recovery when tool calls fail, managing conversation memory and context across long-running interactions, and testing for failure modes that emerge from giving a probabilistic model real-world agency.

This is control systems design meets software engineering. It requires thinking about state management, error handling, concurrency, and safety boundaries. None of this shows up in a data science curriculum.

Evaluating Generative Systems

Here’s where the gap between data science and LLM work becomes most stark.

In traditional data science, evaluation is well-defined. You have a test set. You calculate precision, recall, F1, AUC, RMSE. You know whether the model is good or not. The number tells you.

In LLM work, evaluation is an open research problem. The system produces free-form text. “Correct” is often subjective and context-dependent. The work involves designing evaluation rubrics with multiple dimensions (relevance, faithfulness, coherence, harmlessness, completeness), building LLM-as-judge frameworks where one model evaluates another’s output, running human evaluation protocols and calibrating annotator agreement, testing for hallucination against known ground truth, measuring retrieval quality separately from generation quality, and tracking all of this across thousands of test cases that need to be curated and maintained.

A data scientist knows how to evaluate a model. But “evaluate a model” means something completely different in these two contexts. The methods, the tools, the mindset, and the challenges barely overlap.

Fine-Tuning and Model Adaptation

When teams fine-tune language models — whether through full fine-tuning, LoRA, QLoRA, or RLHF — the process looks nothing like training a random forest on a CSV file. It involves curating instruction-following datasets, understanding tokenization and how it affects training, managing GPU infrastructure that costs thousands of dollars per training run, navigating the tradeoffs between model size and inference cost and quality, evaluating results with methods that borrow from linguistics and cognitive science as much as from machine learning, and understanding alignment techniques that are still being actively researched.

The infrastructure requirements alone put this in a different category. A data scientist working on a laptop with scikit-learn and a Jupyter notebook is operating in a fundamentally different world than someone fine-tuning a 70-billion parameter model across a cluster of A100s.

Tool Integration and Production Systems

Agentic AI systems need to interact with the real world — calling APIs, querying databases, executing code, searching the web, sending messages, updating records. Building these integrations requires designing tool schemas that the model can reliably interpret, implementing authentication and authorization flows, handling rate limiting and error states gracefully, building retry logic and fallback strategies, ensuring the agent doesn’t perform destructive actions without confirmation, and testing the full integration end-to-end under adversarial conditions.

This is software engineering. It’s the same skill set that goes into building any production system that integrates with external services, with the added complexity that the “client” making the API calls is a probabilistic language model that sometimes gets creative with its inputs.

The Venn Diagram Is Almost Two Separate Circles

Let’s compare the core competencies side by side.

A strong data scientist has deep knowledge of statistics and probability, experimental design and causal inference, regression and classification modeling, feature engineering on structured data, SQL and data manipulation, model evaluation with well-defined metrics, and communicating quantitative findings to business stakeholders.

A strong AI/LLM engineer has deep knowledge of transformer architectures and how LLMs process text, prompt engineering patterns and failure modes, RAG system design and information retrieval, agent architectures and tool-use patterns, evaluation of open-ended generative outputs, fine-tuning techniques and training infrastructure, and building production systems that integrate LLMs with external services.

The overlap? They both use Python. They both work with “models” in some sense of the word. They both exist in the general orbit of “AI.” That’s it.

Calling both of these roles “Data Scientist” makes exactly as much sense as calling a cardiac surgeon and a psychiatrist the same role because they both went to medical school and they both help patients. The foundational training overlaps. The actual job does not.

Why This Keeps Happening

The reason is the same as it was a decade ago: organizational laziness, budget constraints, and hype.

Laziness. Creating a new role requires defining it, getting headcount approved, building a career ladder, and figuring out where it sits in the org chart. It’s much easier to take an existing title and stretch it. “Data Scientist” already exists, HR already has the job code, the compensation bands are already set. Just add “GenAI” to the title and post it.

Budget. Hiring an AI engineer and a data scientist is two headcounts. Hiring a “Data Scientist (GenAI)” is one headcount doing two jobs. The math is obvious, and it works right up until the person burns out, the LLM product quality is terrible, and the actual data science work doesn’t get done because nobody has time for it.

Hype. Executives read that AI is going to transform their business. They want to “do AI.” They don’t understand the difference between building an LLM chatbot and analyzing customer churn data — both involve AI, right? So they tell the data science team to “figure out the AI thing” and move on to the next board meeting.

The result is predictable. Data scientists who were hired to do statistical analysis are now debugging LangChain errors and configuring Pinecone indexes. They’re not good at it — not because they’re not smart, but because it’s not what they were trained for. Meanwhile, their actual data science work — the experiments, the models, the analysis — is piling up undone.

The Human Cost

Let’s talk about what this does to actual people, because it’s easy to lose that in the abstraction.

There’s a data scientist right now who spent years developing expertise in Bayesian statistics and causal inference. She took her current job because the description emphasized experimentation and modeling. Six months in, she’s spending 80% of her time trying to get a RAG pipeline to stop hallucinating answers from the wrong documents. She’s not learning transferable data science skills. She’s not doing the work she’s good at. She’s googling “LangChain retriever returns irrelevant chunks” at 11 PM and wondering if she chose the wrong career.

There’s another data scientist who genuinely enjoys the LLM work — he finds it fascinating and wants to build a career in it. But his title says “Data Scientist,” his performance is evaluated against data science metrics, and when he eventually interviews elsewhere, his resume says he was a data scientist who “also did some GenAI stuff.” The market doesn’t know how to value his actual skills because the role didn’t have its own identity.

Both of these people are being failed by the same organizational decision: refusing to acknowledge that this is a different job.

What the Roles Should Actually Be Called

The industry is already converging on better terminology. Companies that are serious about this work have started creating distinct roles:

AI Engineer. The person who builds LLM-powered applications end to end — designing prompts, building RAG systems, integrating tools, shipping features. This is the most common emerging title, and it accurately describes the work as engineering, not science. The AI engineer’s primary output is a working system, not an analysis or a model evaluation report.

AI/LLM Solutions Architect. The person who designs the overall architecture of an AI system — deciding which models to use, how to structure the retrieval pipeline, how agents should be orchestrated, what guardrails are needed, and how the system will scale. This is a senior technical design role.

Prompt Engineer. In complex systems with dozens of prompt-dependent components, designing and maintaining effective prompts is a full-time job. The prompt engineer tests prompt variations systematically, manages prompt versioning, and ensures consistent behavior across the system.

ML Engineer (LLM/GenAI specialization). The person who handles fine-tuning, training infrastructure, model optimization, and deployment of language models specifically. This is machine learning engineering with a specialization — distinct from both traditional ML engineering (which focuses on deploying models trained on structured data) and data science.

AI Product Manager. The person who defines what the AI system should do, prioritizes features, manages the roadmap, and translates business needs into technical requirements for the AI engineering team. This role understands the capabilities and limitations of LLMs well enough to set realistic expectations.

Each of these roles has a distinct skill set, a clear scope of responsibility, and a career trajectory that doesn’t require pretending to be something else. None of them is a data scientist.

Where Data Scientists Actually Fit in the LLM World

None of this means data scientists have no role in organizations building with LLMs. They absolutely do — but in their actual capacity, doing actual data science.

Analyzing system performance. An LLM-powered product generates mountains of interaction data — user queries, model responses, retrieval results, latency metrics, user feedback signals. A data scientist can analyze this data to identify patterns in failure modes, quantify quality across different query types, and surface insights that inform engineering priorities. This is exploratory data analysis, and it’s exactly what data scientists are trained for.

Designing experiments. Should we use GPT-4o or Claude for this use case? Does adding a re-ranking step improve retrieval quality? Do users prefer shorter or longer responses? These are empirical questions that benefit from proper experimental design — randomization, sample size planning, controlling for confounders, statistical testing. A data scientist brings rigor to decisions that would otherwise be made by vibes and anecdotes.

Building evaluation frameworks. While the AI engineer builds the LLM-as-judge pipeline, the data scientist can design the evaluation methodology — sampling strategies, inter-rater reliability measures, statistical tests for detecting quality regressions, and confidence intervals around quality scores. This is measurement science, and data scientists are uniquely qualified for it.

Modeling user behavior. How do users interact with the AI product over time? What predicts retention? What drives users to give negative feedback? These are traditional data science questions applied to a new product surface. The methods (survival analysis, classification, segmentation) are the same even if the product is novel.

The pattern is clear: data scientists add value by doing data science about LLM systems, not by building or operating the LLM systems themselves.

A Practical Test for Hiring Managers

If you’re writing a job description right now and you’re not sure whether the role is a data scientist or something else, here’s a simple test.

Ask yourself: What is the primary output of this role?

If the answer is insights, analyses, experiments, and statistical models that inform business decisions — it’s a data scientist.

If the answer is a working LLM-powered application, agent, or system that users interact with — it’s an AI engineer.

If the answer is both — it’s two roles, and you need to hire two people. Or at minimum, be honest that you’re asking one person to do two jobs, title it accurately, and pay accordingly.

Here’s another test: What does a bad week look like?

If a bad week means an analysis produced a misleading result because of a subtle data quality issue, or an A/B test was designed without enough statistical power — you need a data scientist.

If a bad week means the agent called the wrong API and sent a customer incorrect information, or the RAG pipeline retrieved irrelevant documents and the chatbot hallucinated an answer — you need an AI engineer.

These are not the same failure mode. They’re not the same skill set to fix. They’re not the same job.

The Cost of Getting This Wrong — Again

We already know what happens when companies refuse to properly scope and title technical roles. We watched it play out with data science over the last decade. Unrealistic expectations led to burned-out professionals, failed initiatives, executive disillusionment with the entire field, and years of organizational damage that some companies still haven’t recovered from.

The LLM and agentic AI space is moving faster than data science ever did. The technology is evolving monthly. The best practices are still being discovered. The tools and frameworks are changing under people’s feet. This is exactly the wrong time to saddle the work with a borrowed title and undefined scope.

Companies that get this right — that create distinct AI engineering roles, give them clear mandates, build real career ladders for them, and let their data scientists keep doing data science — will build better products, retain better people, and move faster.

Companies that dump it all into one overloaded “Data Scientist (GenAI/LLM/Agentic AI/Full-Stack/Unicorn)” role will get exactly what they deserve: mediocre everything, exhausted people, and a Slack channel full of unresolved LangChain errors sitting next to an abandoned churn model that nobody had time to finish.

The Call to Action

If you’re a hiring manager: create the right roles. Define them clearly. Hire for the actual work.

If you’re a data scientist being asked to build LLM applications: advocate for yourself. Name the mismatch. Push for your organization to recognize this as a distinct function. You’re not failing if you’re not good at prompt engineering — you were hired to do something else.

If you’re someone who loves building with LLMs and agents: claim your own identity. You’re an AI engineer. You’re building something new. You don’t need to borrow someone else’s title to be legitimate. This field is real, it’s growing, and it deserves its own name.

And if you’re an executive who just wants your company to “do AI” — take thirty minutes to understand that “AI” is not one thing. The person who analyzes your data is not the person who builds your chatbot. Investing in both, separately, with clear expectations, is how you actually get value from either.

Stop asking one person to be five people. We already learned this lesson once. Let’s not learn it again.

If you missed it, this is a companion piece to my earlier article on the core skills of a data scientist — and why the role was already bloated before LLMs entered the chat. That one is called “Stop Asking Data Scientists to Be Five People in a Trench Coat.” Read them together. Then forward both to your HR department.

Brian James Curry is an Enterprise AI Strategist, Data and Marketing Scientist, and Applied Researcher. He is the founder of Vector1 Research, an independent research lab focused on marketing economics, causal systems, cognitive AI architecture, and the economic implications of intelligent systems. He also founded KC AI Lab, a data science community and incubator for applied machine learning education.