The Data Science Death Trap: When Organizations Don’t Understand What Data Science Actually Takes

Why data science roles at companies with poor leadership, inadequate resources, and managers who’ve never done the work might be the most exhausting positions in tech

There’s a particular kind of exhaustion that comes from being a data scientist at a company that doesn’t understand data science. It doesn’t matter if you’re the first hire or joining an existing team — the symptoms are the same. The slow realization that you’ve been hired to be a unicorn. The dawning horror that your manager has never built a model, never cleaned a dataset, never experienced the winding, recursive reality of actual data science work. The mounting frustration as leadership demands “AI transformation” while refusing to invest in infrastructure, realistic timelines, or adequate staffing.

Maybe you joined a team that looked promising on paper — five data scientists, a dedicated budget, executive sponsorship. But once inside, you discovered the team is managed by someone promoted from marketing analytics who thinks a random forest is something in Oregon. The “infrastructure” is a shared Google Drive folder. The executive sponsor wants ChatGPT-level results by Q3 but won’t approve cloud computing costs.

Or maybe you are the first data scientist, brought in to “do AI” or “leverage our data.” What nobody mentioned was that you’d also be writing SQL queries at 11 PM, explaining what a training set is to executives for the third time this quarter, begging IT for database access, building your own data pipelines because there are none, and somehow — somehow — delivering a production-grade recommendation engine by next Tuesday because “it shouldn’t be that hard, right?”

Whether you’re employee number one or number ten on a data science team, the death trap looks the same: unrealistic expectations, inadequate support, managers who can’t evaluate or advocate for your work, and a slow burn toward inevitable burnout.

Welcome to the death trap.

The Numbers Don’t Lie (But Nobody Wants to Hear Them)

Let’s start with the brutal reality that most organizations refuse to acknowledge.

According to RAND Corporation’s comprehensive 2024 study, which interviewed 65 experienced data scientists and engineers, more than 80% of AI projects fail to reach meaningful production deployment — exactly twice the failure rate of IT projects without AI components. This isn’t a fluke. This is systemic.

The situation has actually gotten worse. S&P Global Market Intelligence’s 2025 survey of over 1,000 enterprises found that 42% of companies abandoned most of their AI initiatives this year, up dramatically from just 17% in 2024. The average organization scrapped 46% of AI proof-of-concepts before they reached production.

MIT’s 2025 State of AI in Business report delivers perhaps the most sobering statistic: 95% of generative AI pilots fail, often due to brittle workflows and misaligned expectations. Despite $30–40 billion in enterprise AI investment, only 5% of initiatives are producing measurable returns.

These aren’t just numbers. These are careers. These are burnout stories. These are talented people who were set up to fail.

The Unicorn Problem

The term “data scientist” was coined to describe what DJ Patil and Jeff Hammerbacher were doing at LinkedIn and Facebook in the early 2010s. Thomas Davenport called it “the sexiest job of the 21st century.” What nobody emphasized was that those early data scientists were working at companies with sophisticated data infrastructure, engineering support, and leadership that understood what data science actually required.

Today, most organizations have data scientists — but having data scientists and understanding data science are two very different things. The Harvard Data Science Review published research confirming what frustrated data scientists already know: organizations are still searching for “unicorns” — professionals expected to master machine learning, numerical computing, data management, statistical inference, distributed programming, business communication, and stakeholder management all at once.

As the researchers put it bluntly: “Unicorns are rare. In fact, they may not exist.”

A 2025 study in Business & Information Systems Engineering noted that this “lack of clarity leads to the misconception that one person, the so-called data science unicorn, can do it all.” The study found that data science has become a team sport — requiring data engineers, ML engineers, data analysts, and product managers working in collaboration.

But even organizations with entire data science teams often structure them poorly. They hire data scientists without hiring data engineers. They put non-technical managers in charge of technical teams. They create “centers of excellence” that have no connection to business units. They build teams of five people and expect them to deliver what would take fifty at a mature tech company.

The unicorn problem isn’t just about individual job descriptions — it’s about organizational delusion. Companies believe that hiring smart people with “data scientist” in their title will magically produce AI transformation, without investing in the infrastructure, leadership, and realistic expectations that make data science actually work.

What Data Science Actually Is (And Isn’t)

Before we go further, we need to address a fundamental confusion that underlies the entire death trap: most organizations don’t understand what data science actually is.

Data science is science. The clue is in the name.

It is a discipline focused on discovery — extracting insights, identifying patterns, testing hypotheses, and creating value through advanced statistical methods, machine learning, and mathematical abstractions. A data scientist’s core function is to answer questions that haven’t been answered before, to find signal in noise, to transform raw data into knowledge that drives better decisions.

But here’s what many organizations miss: data science often isn’t even about finding answers. It’s about finding the questions.

When you mine data properly, you don’t always know what you’re looking for. You explore. You notice anomalies. You ask “why is this happening?” and “what would happen if?” You follow threads that lead nowhere and threads that lead somewhere unexpected. The path is winding, recursive, and non-linear. You might start investigating customer churn and discover a pricing problem. You might start building a recommendation engine and uncover data quality issues that, once fixed, improve every model in the company.

This exploratory, question-finding work is where much of the real value of data science lies — and it’s almost impossible to put on a project timeline. How do you schedule discovery? How do you estimate when you’ll find something you didn’t know existed?

Organizations that treat data science as “answer the question we already asked” miss the bigger opportunity. The best data scientists don’t just answer the question — they find the question that should have been asked in the first place.

What data science is not:

Data science is not data engineering. Data engineers build and maintain the pipelines, warehouses, and infrastructure that make data accessible. They ensure data flows from source systems into usable formats. This is critical work — but it’s engineering, not science.

Data science is not DevOps or MLOps. Deploying models to production, managing containers, setting up CI/CD pipelines, monitoring system performance — this is operations work. It requires a completely different skill set focused on reliability, scalability, and system administration.

Data science is not software engineering. Writing production-grade code, building APIs, creating user interfaces, maintaining codebases — this is software development. Data scientists may prototype, but turning prototypes into production systems is engineering.

Data science is not AI tool building. Integrating third-party AI services, fine-tuning pre-built models, building chatbots with off-the-shelf frameworks — this is implementation work. It may use AI, but it’s not science.

Data science is not business intelligence. Creating dashboards, generating reports, tracking KPIs — this is analytics. It’s valuable, but it’s descriptive, not predictive or prescriptive.

The scientific method involves hypothesis generation, experimentation, iteration, and — crucially — the possibility of failure. When a scientist runs an experiment and discovers their hypothesis was wrong, that’s not a failure. That’s learning. That’s how science works.

This is fundamentally different from engineering, where you start with known requirements and build toward a defined outcome. Engineers are expected to deliver working systems on predictable timelines. Scientists are expected to discover what’s possible — and sometimes the answer is “this approach doesn’t work.”

As MIT Sloan noted, “managing a data science research effort tends to be a dynamic and self-correcting process; it is difficult to plan precisely either a project’s timing or final outcomes.” This isn’t a bug. It’s an inherent feature of research work.

When organizations hire data scientists but expect engineers, the mismatch is catastrophic. They want predictable deliverables on fixed timelines. They get frustrated when experiments don’t immediately produce results. They don’t understand why “building a model” takes months instead of days.

One data scientist captured this perfectly: “Jobs that throw my work away are not satisfying jobs — and it indicates that the company doesn’t really know what to do with my work.” When organizations treat data science as implementation rather than research, they miss the entire point.

The proper organizational model separates these functions:

Data Engineers build and maintain data infrastructure
Data Scientists conduct research, build prototypes, and discover insights
ML Engineers productionize models and maintain production systems
Analytics Engineers create dashboards and reporting
Software Engineers build applications and user-facing systems

When data scientists are expected to do all of these jobs — or when teams are staffed without the supporting roles — they can do none of them well. And yet that’s exactly what many organizations expect, whether they’re hiring their first data scientist or building out a team of twenty without adequate engineering support.

The death trap exists, in large part, because companies want the prestige of “doing AI” without understanding that real data science is research — uncertain, iterative, and impossible to schedule like a software sprint.

The 80% Tax

Here’s where the job description meets reality.

The widely-cited statistic that data scientists spend 80% of their time on data preparation — cleaning, organizing, and wrangling data — has been repeated so often it’s become cliché. But the research backs it up.

Anaconda’s State of Data Science survey found that data scientists spend approximately 45% of their time on data preparation tasks alone, with data cleansing accounting for over a quarter of an average workday. The Forbes/CrowdFlower survey put it higher: 60% of time spent cleaning and organizing data, with another 19% spent collecting datasets. Only 20% of their time goes to actual analysis — the work they were hired to do.

But here’s what those statistics don’t capture: the 80% tax gets even worse in organizations without proper infrastructure. Even established data science teams suffer when there’s no data engineering support, no maintained pipelines, no reliable data warehouse. The data you need is scattered across seventeen different systems, half of which you don’t have access to, and the other half are maintained by people who don’t know why you need it.

As one data scientist described on Built In: “If a data scientist is brought on before their time, they’ll likely end up doing a lot of engineering work, just to get the data to a place where it can actually provide insights.” This applies whether you’re the first hire or joining a team of twenty — if the infrastructure isn’t there, you’re doing data engineering whether you signed up for it or not.

The Timeline Trap

The median time to deploy a single machine learning model, according to Algorithmia’s research, is 8 to 90 days — and that’s for organizations with existing ML infrastructure. For organizations without mature ML programs, the timeline extends to 6 to 12 months or more from concept to production.

Let me repeat that: 6 to 12 months. For one model. With a mature team.

Now consider what happens when a company with no data science experience hires their first data scientist and asks for a “quick win” in two weeks.

The disconnect is staggering. McKinsey’s 2025 AI survey found that organizations seeing “significant” financial returns from AI are twice as likely to have redesigned end-to-end workflows before selecting modeling techniques. But most companies skip this step entirely. They want the magic of AI without the infrastructure to support it.

Informatica’s CDO Insights 2025 survey identified the top obstacles to AI success: data quality and readiness (43%), lack of technical maturity (43%), and shortage of skills (35%). Winning programs allocate 50–70% of their timeline and budget to data readiness — extraction, normalization, governance, and quality assurance. Most organizations allocate approximately zero.

The Burnout Epidemic

Perhaps unsurprisingly, data professionals are burning out at alarming rates.

A 2021 survey by 365 Data Science found that 97% of the 600 data engineers surveyed experienced burnout in their day-to-day job. Seventy-nine percent were considering leaving the industry entirely. The primary causes? Unrealistic expectations, lack of organizational understanding of their role, and restrictive data governance policies that limit their ability to actually do their work.

The TDWI report put it starkly: “At the root of that stress is a problem that has become pervasive in our industry: the lack of communication between business leadership and engineering teams.”

Data scientists stay with their current employer for only 1.7 years on average. That’s not a career path — that’s a revolving door.

DHR Global’s 2025 survey found that 82% of white-collar knowledge workers reported being “slightly” to “extremely” burned out. For data scientists specifically, the combination of unrealistic expectations, isolation, and the constant pressure to deliver magic makes burnout almost inevitable.

The Scope Creep Spiral

When your organization doesn’t understand data science, everything adjacent to data becomes your job — whether you’re the only data scientist or one of many on an under-resourced team.

Marketing needs a dashboard? That’s you. Finance wants forecasting? You. Sales needs lead scoring? Also you. The CEO read an article about ChatGPT on a plane and now wants to know why we don’t have one? Guess who’s explaining transformer architectures in the next all-hands?

In dysfunctional organizations, data scientists become the data engineer writing ETL pipelines because there isn’t one. The data analyst building dashboards because leadership needs them yesterday. The ML engineer deploying models because who else would do it? The project manager because someone has to set expectations. The educator because nobody else understands what you do. The evangelist because if you don’t explain the value of your work, nobody else will.

As one experienced data scientist wrote: “If you’re the only data scientist, your company will consider you the authority on data science. You’ll be expected to both generate ideas (understand the business context) and carry them out (know the technical execution).” But this same dynamic plays out on larger teams too — when leadership doesn’t understand the work, they don’t understand how to staff it properly.

This is not a job. This is five jobs wearing a trench coat pretending to be one job.

The Support Structure That Doesn’t Exist

Research consistently shows that successful data science requires a support ecosystem. You need a data champion in business leadership who can advocate for your work. You need engineering support to productionize models. You need data engineering to build and maintain pipelines. You need product management to align work with business goals.

Without this support, as one researcher wrote, “your work can easily be unseen, unnoticed, and undervalued, and you will constantly need to fight to prove the value you’re bringing.”

The RAND study identified five root causes of AI project failure:

Misunderstanding the problem: Stakeholders don’t understand — or miscommunicate — what problem actually needs solving
Lack of necessary data: Organizations don’t have the data to adequately train an effective model
Technology over solutions: Companies focus on using the latest technology rather than solving real problems
Missing technical infrastructure: No frameworks for MLOps, deployment, or monitoring
Skill gaps: Lack of talent to build, deploy, and maintain AI systems

Notice that only one of these is about technical skill. The rest are organizational failures — and when you’re the solo data scientist, organizational failures become your personal problem to solve.

The Accidental Manager Problem

There’s another layer to this death trap that doesn’t get discussed enough: the people managing data scientists often have no idea what data science actually involves.

MIT Sloan Management Review identified this as a fundamental challenge: “Because many [managers] aren’t trained in data science, many managers aren’t easily able to evaluate the technical details of a project; as a result they aren’t able to judge the quality of the research — or determine whether a project should take as long as it does.”

This creates a cascading failure. A manager who doesn’t understand data science can’t set realistic timelines, can’t properly evaluate work quality, can’t advocate effectively for resources, and can’t protect their team from unrealistic demands from above. They’re essentially flying blind while being held accountable for results they don’t understand.

The problem often starts with how companies hire data science managers. As Data Science PM documented, companies tend to fall into two traps: they either hire great data scientists who have no management experience, or they hire experienced managers who have no technical understanding. Both approaches fail.

One job posting analyzed by researchers looked for extensive technical qualifications but never mentioned managing people, managing projects, or experience working within a data science team. Another posting valued management experience but required no data science knowledge whatsoever. The result? Managers who either can’t lead people or can’t understand the work they’re managing.

Harvard Business Review put it bluntly: “Many managers of data science teams become managers because they were great individual contributors and not necessarily because they have the skills or training to lead a team. But management is a skill unto itself, and relying on your experience as a successful individual contributor is not enough.”

And here’s the kicker: “Great data scientists have career options and won’t abide bad managers for very long.” That 1.7-year average tenure? Bad management is a significant driver.

Senior data scientist Peadar Coyle identified three critical pitfalls of non-technical managers leading data science teams:

The “Magic Quickly” Expectation: Without hands-on experience building data products, managers don’t understand why things take time. As one data science consultant noted, if no one has looked at the data before, add six months to any project estimate. Non-technical managers hear this and think you’re being difficult, not realistic.

Missing Technical Nuance: Machine learning is complicated. Statistics is complicated. There are dozens of ways a model can fail — violated assumptions, selection bias, overfitting, data leakage — that a non-technical manager simply can’t evaluate. When your manager can’t tell the difference between a well-built model and a broken one, how do they know if you’re doing good work?

The Communication Disconnect: There exists a considerable communication gap between technical specialists and management on ML projects. Data scientists speak one language; their managers speak another. Without someone who can bridge that gap, projects drift, expectations diverge, and failure becomes inevitable.

The worst scenario? When someone from an adjacent function — marketing analytics, business intelligence, IT — gets promoted to “manage the data science initiative” because they’re the closest thing to technical the company has. They inherit a function they don’t understand, are held accountable for outcomes they can’t evaluate, and are expected to set strategy for a field they’ve never worked in.

This creates a particular kind of hell for the data scientists reporting to them. You’re explaining basic concepts constantly. You’re defending timelines that should be obvious to anyone who’s done this work. You’re watching your manager commit to deliverables that aren’t possible and then being blamed when those commitments aren’t met. You’re doing the emotional labor of managing up — constantly educating your boss — while also doing your actual job.

As MIT Sloan noted, “Managing a data science research effort tends to be a dynamic and self-correcting process; it is difficult to plan precisely either a project’s timing or final outcomes. For those unused to this type of work, this process can seem quite messy.”

For a manager who came up through traditional project management — where timelines are predictable and deliverables are well-defined — this “messiness” looks like incompetence. It’s not. It’s the inherent nature of research work. But explaining that to someone who’s never done research is exhausting.

Domino Data Lab’s field guide for data science managers acknowledged the fundamental challenge: “Being a technical whiz doesn’t necessarily prepare you to manage others.” But the reverse is equally true: being a great people manager doesn’t prepare you to lead technical work you don’t understand.

The solution isn’t simple. Companies need managers who have both technical credibility and management skills — but that combination is rare. What typically happens instead is that companies hire whichever they can find and hope the missing skills will somehow develop on the job.

They usually don’t. And the data scientists pay the price.

The “Just Use AI” Delusion

Perhaps nothing captures the death trap better than the expectation that AI should be easy.

Executives read about ChatGPT and assume machine learning is plug-and-play. They see demos and assume production deployment is trivial. They hear about “AI transformation” and assume it’s a switch you flip.

The reality? According to Lumenalta’s analysis, organizations with proper ML development processes see 15–30% improved model accuracy, up to 50% reduced training time, and 80% reduction in production issues — but only after investing heavily in data preparation, infrastructure, and quality assurance frameworks.

The Stanford AI Index 2025 reports that only 10–20% of AI models actually reach production, with 90% of initiatives stuck at the experimental stage. The gap between “built a model in a Jupyter notebook” and “deployed a reliable system that creates business value” is enormous — and most organizations vastly underestimate it.

What Actually Takes Time

For those setting expectations — or trying to reset them — here’s what ML development actually involves:

Problem Definition (2–4 weeks) Before writing a single line of code, you need to understand the business problem, define success metrics, assess data availability, and evaluate feasibility. Skip this step, and you’ll build something nobody needs.

Data Collection and Preparation (4–12 weeks) Finding, accessing, cleaning, transforming, and validating data. This includes dealing with missing values, inconsistent formats, duplicate records, and data quality issues. For most organizations, this is the longest phase — and it’s not something you can rush.

Exploratory Analysis and Feature Engineering (2–6 weeks) Understanding patterns in the data, creating features that will help the model learn, and identifying potential issues before they become problems in production.

Model Development and Evaluation (4–8 weeks) Building, training, testing, and iterating on models. This is the part that gets all the attention — but it’s only a fraction of the total effort.

Deployment and Integration (4–12 weeks) Taking a model from a notebook to a production system. This includes building APIs, setting up monitoring, creating rollback procedures, and integrating with existing systems.

Monitoring and Maintenance (Ongoing) Models drift. Data changes. Systems break. Production ML requires continuous monitoring and periodic retraining.

Total timeline for a single end-to-end project: 4–12 months. With a team. With infrastructure.

When a company without data science experience asks for results in two weeks, they’re not being aggressive — they’re being delusional.

Escaping the Trap: Interview Questions That Reveal the Truth

Whether you’re considering your first data science role or your fifth, the interview process is your best opportunity to identify warning signs. Dysfunctional data science organizations share common patterns, and the right questions can expose them before you sign an offer letter. As data science career experts Jacqueline Nolis and Emily Robinson documented in their research on red flags, companies tend to show none of these warning signs or most of them — with few in the middle. Ask the right questions and you can avoid years of frustration.

Questions About Infrastructure and Data Readiness

“What is your data infrastructure like and who maintains it?” If the answer is vague, or if there’s no dedicated data engineering function, you will be doing that work yourself. Follow up: “What format is data typically in — Excel, SQL databases, data warehouse, or csv files scattered across shared drives?”

“How does the company balance complex AI approaches with foundational work like data quality, cleaning, and logging?” This question, recommended by Nolis and Robinson, exposes whether the company understands the hierarchy of data needs. If they’re excited about AI but have no data infrastructure, you’ll be blamed when their unrealistic expectations meet reality.

“Can you walk me through a recent data science project from inception to deployment?” Listen for whether they mention data preparation challenges, iteration, and production deployment — or whether they skip straight to “we built a model.” If deployment isn’t mentioned, their models probably aren’t making it to production.

“What percentage of your data science projects have made it to production in the past year?” This is a direct question that cuts through hype. Industry-wide, the answer is often below 20%. If they can’t answer or seem uncomfortable, that tells you something.

Questions About Team Structure and Support

“Who would I be working with on a typical project?” You’re listening for whether there’s engineering support for deployment, data engineering for pipelines, and product management for prioritization — or whether you’ll be doing all of it yourself.

“What steps does the team take for QA and peer review?” Mature teams have code reviews, practice presentations, and consistent check-ins. If mistakes aren’t caught until work is delivered, someone gets blamed — and it’s usually the data scientist.

“How do you share code amongst the team?” If the answer is “shared network folders” or “it mostly lives on individual laptops,” run. No version control means no collaboration, no accountability, and no way to maintain work when people leave.

“Who does the data science team report to, and what’s their background?” This directly assesses the manager problem. If your manager has no data science experience, ask how they evaluate work quality and set realistic timelines. If the team reports into IT, marketing, or finance with no technical data science leadership, that’s a structural problem that will affect everything.

Questions About Expectations and Culture

“What does success look like for this role in the first six months? The first year?” Unrealistic expectations reveal themselves here. If they expect production models in month one, or “AI transformation” in six months, expectations are wildly misaligned with reality.

“How was it decided that the last few projects should be built?” This question, suggested by experienced data scientists, reveals the prioritization process. If the answer is “the CEO wanted it” or “[executive] said we needed it,” there’s no real process — just reactive chaos.

“What happens when a project takes longer than expected?” You’re gauging whether the culture allows for the inherent uncertainty of research work, or whether delays are treated as failures. The honest answer should acknowledge that timelines in data science are estimates, not commitments.

“Can you tell me about a data science project that failed here? What happened?” This is revealing. Companies that can’t name a failure either don’t do enough work to have any, or have a culture where failure isn’t discussed — which means you’ll be blamed, not supported, when things don’t work.

“What’s the typical tenure of data scientists on your team?” If people are leaving after a year, that tells you something. If they dodge the question, that tells you more.

Red Flags in Their Questions

Pay attention to what they ask you, too:

No time for your questions: They’re not interested in whether you’re a good fit — only in filling a seat.
Inconsistent answers across interviewers: Ask the same question to multiple people. Inconsistency reveals dysfunction.
Overselling the role: As Domino Data Lab’s research noted, “half of data scientists stay at their jobs for two years or less.” If they’re promising paradise, be suspicious.

The Absurdity of Memorization-Based Coding Interviews

While we’re discussing interviews, let’s address the elephant in the room: the continued insistence on LeetCode-style memorization tests in an era where AI tools can write production-grade code.

Many companies still run data scientists through whiteboard coding exercises that test whether you’ve memorized the implementation of binary search trees or can recall the exact syntax for a dynamic programming solution under pressure. You’re expected to write perfect code on a whiteboard or in a Google Doc, without access to documentation, autocomplete, or the tools you’d actually use on the job.

This was always a questionable proxy for job performance. Now it’s absurd.

In 2025, every working data scientist has access to AI coding assistants that can generate, debug, and optimize code in seconds. GitHub Copilot, Claude, ChatGPT, and dozens of other tools have fundamentally changed how code gets written. The skill that matters isn’t whether you’ve memorized sorting algorithms — it’s whether you can effectively direct these tools, evaluate their output, understand their limitations, and integrate their suggestions into production-quality systems.

Testing whether someone can implement quicksort from memory tells you nothing about whether they can architect a machine learning pipeline, debug a data quality issue, or translate a business problem into a technical solution. It tests memorization and performance under artificial pressure — skills that have almost no correlation with actual data science work.

The companies still running these interviews are telling you something important about their culture: they value theater over substance, tradition over effectiveness, and hazing rituals over actually evaluating job-relevant skills. They’re also likely to be behind on adopting AI tools in their actual work, which means you’ll be fighting to use modern tooling while competitors speed past.

When you encounter a company that tests your ability to reverse a linked list on a whiteboard, ask yourself: is this an organization that understands what data science work actually looks like in 2025? Or are they running an interview process designed for software engineers in 2010, applied blindly to a field they don’t understand?

Better interviews assess what actually matters: Can you explore a dataset and identify issues? Can you explain your analytical approach to a non-technical stakeholder? Can you evaluate whether a model is appropriate for a business problem? Can you work through an ambiguous problem and identify what questions need to be answered? These are the skills that determine success. Whether you remember the time complexity of heapsort does not.

Survival Guide: When You’re Already Trapped

Maybe you skipped the interview questions. Maybe you asked and they lied. Maybe the team looked functional until your competent manager left and was replaced by someone who doesn’t understand the work. Maybe the company reorganized and suddenly your well-resourced team reports to someone whose last data experience was Excel pivot tables in 2015. Whatever the reason, you’re now in the death trap: working in a data science function with inadequate leadership, unrealistic expectations, and insufficient support.

Here’s how to survive — and eventually escape.

Accept the Reality

First, acknowledge the situation clearly to yourself. Your manager cannot help you with technical problems because they don’t understand them. They cannot set realistic timelines because they don’t know what realistic looks like. They cannot advocate effectively for you because they can’t explain your work to others. This isn’t going to change through wishful thinking.

This isn’t necessarily your manager’s fault — they may have been put in an impossible position too. But understanding the limitation is essential. You cannot fix their knowledge gap. You can only work around it.

Document Everything

Start keeping records. Document what you’re asked to do, when you’re asked to do it, and what timeline you’re given. Document what you actually do, how long it takes, and why. Document when you raise concerns about timelines or scope.

This isn’t about building a legal case (though it might help). It’s about protecting yourself when things go wrong. When your manager commits to something impossible and it doesn’t happen, you need a record showing you flagged the issue. Email is your friend — follow up verbal conversations with “just to confirm what we discussed” summaries.

Translate Everything Into Business Language

Your manager may not understand t-tests, but they understand money, time, and risk. Learn to translate.

Don’t say: “The model needs more training data and the feature engineering is incomplete.” Say: “We have two options: launch now with 60% accuracy, which means 4 out of 10 predictions will be wrong and could cost us $X in bad decisions, or take three more weeks to improve accuracy to 85%.”

Don’t say: “The data pipeline is broken.”
Say: “We can’t make any predictions until IT fixes the data feed, which they estimate will take two weeks. Until then, this project is blocked.”

Make the business implications explicit. Managers who don’t understand technical work often do understand revenue, cost, and risk.

Manage Up Relentlessly

Since your manager can’t evaluate your work, you need to make it visible and understandable. This means:

Weekly status updates that translate technical progress into business terms. Not “trained three models,” but “tested three approaches and found one that predicts customer churn with 75% accuracy, which would identify $X in at-risk revenue.”

Explicit timeline communication. When given a project, immediately respond with a written estimate that breaks down phases: data acquisition (X weeks), exploration (X weeks), modeling (X weeks), testing (X weeks), deployment (X weeks). Make the complexity visible.

Preemptive problem flagging. Don’t wait until you miss a deadline to mention the problem. Flag risks early and in writing: “I want to note that this timeline assumes we have clean data. If data quality issues emerge, timeline could extend by 2–4 weeks.”

Regular calibration. Every few weeks, explicitly check whether your manager’s expectations match reality: “Just want to make sure we’re aligned — my understanding is the priority is X, and the timeline is Y. Is that still accurate?”

Find Allies

You need people in the organization who understand your work well enough to advocate for you. This might be:

An engineering lead who understands technical complexity
A product manager who’s worked with data scientists before
Another technical IC who can validate that your timelines are reasonable
A senior leader who’s data-literate enough to be a champion

Invest in these relationships. Explain your work to them. Help them with their problems. When your manager makes unrealistic commitments, these allies can provide air cover or at least validation that you’re not crazy.

Set Boundaries Explicitly

When asked to do something impossible, don’t just say “I’ll try.” Name the tradeoff:

“I can do X or Y by that deadline, but not both. Which is the priority?”

“I can hit that timeline if we reduce scope to Z. Otherwise, realistic timeline is W.”

“I can start this new project, but it will pause the current one. Which should I prioritize?”

Force decisions. Don’t let ambiguity set you up to fail at everything by trying to do everything.

Build Your Escape Route

While you’re surviving, prepare to leave. Keep your skills sharp — work on side projects, contribute to open source, take courses. Keep your resume updated. Maintain your network — talk to former colleagues, attend meetups, stay visible in the community.

When you interview for your next role, you’ll know exactly which questions to ask. You’ve learned what the death trap looks like from the inside.

Know When to Leave

There are situations that don’t improve. If:

Your manager actively takes credit for your work or blames you for their mistakes
The organization fundamentally doesn’t value data science and won’t invest in it
You’re experiencing health impacts from stress — insomnia, anxiety, depression
You’ve tried managing up and setting boundaries and nothing changes
The culture punishes people for speaking honestly about problems

Then the right answer is to leave. Not every situation is salvageable. Some organizations need to fail at data science several times before they learn — and you don’t have to be the person they fail with.

The average data scientist stays 1.7 years. That’s not because data scientists are flaky. It’s because many organizations are not ready for data science, and talented people eventually recognize when they’re in an impossible situation.

Leaving isn’t failure. Sometimes it’s the smartest thing you can do.

A Note for Organizations

If you’re an executive or hiring manager reading this, here’s what you need to understand:

Put technical leaders in charge of technical teams. Data science teams need managers who understand data science — not marketing directors, IT managers, or business analysts who got promoted because they were “closest to data.” Would you put someone who’s never practiced law in charge of your legal department? Then why put someone who’s never built a model in charge of your data scientists?

Don’t hire data scientists before you’re ready for them. You need data pipelines before you need models. You need clean, accessible data before you need algorithms. Hiring data scientists before your data is ready is like hiring a chef before building a kitchen.

Staff appropriately. One data scientist cannot do the work of a data engineer, ML engineer, data analyst, and product manager combined. If you can only afford one hire, be realistic about what that one person can accomplish. If you have a team of five, don’t expect the output of a team of fifty.

Set realistic timelines. Six months to first production model is aggressive, not conservative. The industry average for deploying a single model is 8–90 days with mature infrastructure, and 6–12 months without it. Plan accordingly.

Listen to your data scientists when they tell you what’s possible. They’re not being difficult — they’re being realistic. When they say a project will take six months, they’re not padding the estimate. When they say the data isn’t ready, they’re not making excuses.

If you’re managing data scientists without technical experience, acknowledge the limitation. Read, learn, ask questions, and most importantly — defer to their expertise on technical matters, just as you’d defer to a lawyer’s expertise on legal matters. Your job is to clear organizational obstacles, not to evaluate whether their model architecture is correct.

Stop using outdated interview processes. LeetCode-style memorization tests don’t predict data science job performance. They test skills that AI tools have made largely irrelevant while failing to assess the skills that actually matter. Design interviews that evaluate real job-relevant capabilities.

The 80%+ failure rate for AI projects isn’t inevitable. It’s the result of organizational dysfunction that’s entirely within your power to fix. But it requires humility, investment, and patience — three things that are often in short supply when executives are excited about AI.

The Bottom Line

The data science death trap exists because of a fundamental mismatch between expectations and reality. Organizations want AI magic. What they need is infrastructure, realistic timelines, competent leadership, appropriate staffing, and patience.

The 80%+ failure rate for AI projects isn’t because data scientists are bad at their jobs. It’s because organizations are bad at understanding what the job actually requires — and then they compound that failure by putting people in charge who don’t understand it either.

If you’re a data scientist in this situation, know that the struggle isn’t personal. The burnout you’re feeling isn’t a failure of character — it’s a predictable outcome of organizational dysfunction. You’re not bad at your job; you’re in a job that’s been set up to fail.

And if you’re an organization wondering why your “AI initiative” isn’t delivering results, consider this: your data scientists might not be underperforming. They might be drowning — under-resourced, poorly managed, and held to expectations that have no basis in reality.

The death trap is real. But it’s not inevitable. With realistic expectations, proper support, competent leadership, and a willingness to invest in the unglamorous work of data infrastructure, organizations can actually succeed with data science.

The first step is acknowledging that building production ML systems is genuinely hard — and that no amount of executive enthusiasm can substitute for the expertise, resources, and time it actually takes.

Sources

RAND Corporation. “The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed.” August 2024.
S&P Global Market Intelligence. “Enterprise AI Adoption Survey.” 2025.
MIT Sloan. “The GenAI Divide: State of AI in Business 2025.”
McKinsey & Company. “The State of AI in 2025.”
Informatica. “CDO Insights 2025 Survey.”
Stanford University. “AI Index Report 2025.”
365 Data Science. “Data Engineer Burnout Survey.” 2021.
Anaconda. “State of Data Science 2020.”
Harvard Data Science Review. “From Unicorn Data Scientist to Key Roles in Data Science: Standardizing Roles.” Summer 2022.
Business & Information Systems Engineering. “Beyond the Unicorn? Job Roles in Data Science.” July 2025.
DHR Global. “Workforce Burnout Survey.” 2025.
Algorithmia. “State of Enterprise Machine Learning.” 2020.
TDWI. “How to Fix Data Engineer Burnout.” 2023.
MIT Sloan Management Review. “Why Managing Data Scientists Is Different.” Roger M. Stein.
Harvard Business Review. “Managing a Data Science Team.” October 2018.
Data Science PM. “6 Actions to Be a Better Data Science Manager.” January 2025.
Domino Data Lab. “Field Guide: Managing Data Science Teams.” November 2024.
Peadar Coyle. “Three Pitfalls for Non-Technical Managers Managing Data Science Teams.” 2017.
MIT Sloan. “6 Steps for Leading Successful Data Science Teams.” August 2021.
Jacqueline Nolis & Emily Robinson. “Red Flags in Data Science Interviews” and “Build a Career in Data Science.”
Hooked on Data. “Red Flags in Data Science Interviews.” 2018.
Dataquest. “Common Workplace Problems for Data Scientists, and How to Address Them.” March 2023.
Analytics Vidhya. “5 Key Reasons Why Data Scientists Are Quitting Their Jobs.” September 2020.
Harvard Business Review. “Are Your Company’s Leaders and Data Scientists on the Same Page?” February 2020.
Towards Data Science. “How to Spot Red Flags in a Data Science Job Opportunity.” January 2025.
AltexSoft. “How to Structure a Data Science Team: Key Models and Roles.” June 2020.

If this resonated with you, you’re not alone. The first step out of the death trap is recognizing you’re in one.

About the Author: Brian Curry is the creator of PyCausalSim, Papilon and an AI, Data, and ML Scientist, analytics and Data Science professional with over 20 years of experience across enterprise companies, and founder of Vector1 AI, a part time side project developing tools and systems to move from correlation-based analytics to causal decision-making.