The Data Access Layer Is Where Agent Projects Actually Stall

"The Model Is the Easy Part" — And Why That's Literally True

Swapping the model behind an agent is an afternoon of work. Change the client, re-run your eval harness, compare the scores, ship. If you've built the eval harness that lets you do that safely, model choice becomes a tuning decision, not an existential one. Prompting is similar — most of the gains come in the first day, and the rest is diminishing returns you measure your way through.

Now try to get that same agent to answer a question that requires joining a customer's billing records to their support history, where the billing data lives in a system with no API, the support data is in a warehouse refreshed nightly, and the agent must never surface a row belonging to a different tenant. None of that is a model problem. All of it has to be solved before the model does anything useful, and every piece of it touches a system owned by a team that didn't ask for your project.

We've watched genuinely strong engineering teams burn three-quarters of a delivery window here while the LLM sat idle behind a feature flag. The model wasn't the constraint. It was never the constraint.

The model wasn't the constraint. It was never the constraint.

The Four Things That Actually Eat the Calendar

Across our engagements, the data access layer fails in four recognisable ways. None of them are exotic. All of them are boring enough that they get waved through in scoping and then detonate in week three.

• →Stale ETL. The agent confidently answers from data that's a day — or a quarter — out of date, because the pipeline feeding its retrieval store runs on a schedule nobody documented. Users catch it faster than any eval will.
• →Inconsistent schemas. The same concept — a customer, an order, an account — is modelled three different ways across three systems, and reconciling them into something the agent can reason over is a data-engineering project wearing an AI costume.
• →Row-level permissions. The agent has to respect the exact access rules a logged-in human would get: this user sees these accounts, this role sees these fields. Getting that right, per query, is the single most underestimated task in enterprise agent work.
• →No clean API. The authoritative data lives behind a legacy system that only ever exposed a UI. Now someone has to build and own an integration layer that didn't exist, on a timeline that assumed it did.

Where the Weeks Really Went: Identity and Tenant Isolation

The one that consistently surprises teams is identity. An agent is not a user. It's a service that acts on behalf of many users, often across many tenants, and it holds credentials broad enough to be dangerous. If you give it a system-wide read token to make the demo work — and almost everyone does, to make the demo work — you've built a data-leak engine that happens to also answer questions.

In one enterprise engagement, reconciling the agent's data access with the customer's existing role-based access controls so it could not surface another tenant's data across roles took longer than building the agent's actual reasoning loop. The hard part wasn't the policy — the customer already had a clean RBAC model for their app. The hard part was propagating the caller's identity all the way through the agent, through every tool call and every retrieval, so that a vector search couldn't quietly return a chunk the calling user was never allowed to see. RAG indexes flatten your permission model unless you deliberately rebuild it inside them.

We now treat every agent as an untrusted insider rather than a trusted system: it inherits the permissions of whoever invoked it, nothing more, and every data path is filtered by that identity before the model ever sees a token. That framing turns a vague security worry into concrete, testable plumbing — and it's plumbing you build before the interesting part, not after.

The Pre-Kickoff Checklist That De-Risks the Whole Thing

Before you commit a timeline to an agent project, run the data access layer through these questions. If you can't answer them, that's your first sprint — and you should scope it explicitly rather than discovering it inside another workstream. We fold this into the shape of a project that actually ships precisely so it doesn't ambush week three.

Every 'no' or 'we're not sure' here is a week you haven't budgeted yet. Better to find them now, on a whiteboard, than in a status meeting when the pilot was due.

• →Identity propagation: Can we pass the calling user's identity through every tool call and retrieval, and enforce their permissions at each step — not just at the front door?
• →Tenant isolation: Is there a hard boundary that makes it structurally impossible for the agent to return one tenant's data to another, and can we test it adversarially?
• →Data freshness: What's the actual refresh cadence of every source the agent reads, and does the task tolerate that staleness? Who owns each pipeline?
• →Schema reconciliation: How many representations of the core entities exist across systems, and who is doing the join? Is that a day or a month of work?
• →API surface: Does every authoritative source expose a callable interface, or are we building and owning integrations that don't exist yet?
• →Audit and revocation: Can we log exactly what data the agent accessed on whose behalf, and cut its access fast if something leaks?

Why This Keeps Getting Skipped

Model demos are seductive and cheap. You wire an LLM to a sample dataset, it answers beautifully, and everyone concludes the project is 90% done. The 90% that's left is the data access layer, and it doesn't demo — there's no screenshot for 'we correctly returned nothing because this user isn't allowed to see it.'

There's also an ownership gap. The AI team can build the agent, but they don't own the billing system, the warehouse, or the RBAC model. Those belong to teams with their own roadmaps, and the integration work sits in the seams between them. This is one of the organisational reasons pilots die around month four: the technical work was tractable, but the cross-team plumbing had no single owner and no budget line.

Be honest with yourself about which problem you're actually solving. If your bottleneck is genuinely the reasoning — a novel task the current models can't do — that's rare and interesting. Far more often the bottleneck is that your data is spread across seven systems with inconsistent permissions and no clean way in. That's not an AI problem. It's the problem the AI project is going to have to solve first, whether you scoped it or not.

What's Still Genuinely Hard

I don't want to pretend the checklist makes this easy. Permission-aware retrieval is still a young discipline — filtering a vector store by row-level access at query time, without wrecking latency or recall, involves real trade-offs and there's no clean off-the-shelf answer for every stack. Legacy systems with no API sometimes leave you screen-scraping, and that's as fragile as it sounds.

And freshness-versus-cost is a live tension: the more current you keep the agent's data, the more you pay in pipeline complexity and infrastructure, and the task's actual tolerance for staleness is often only clear once real users hit it. These are open problems we manage rather than solve outright.

The data access layer isn't impossible. But it's the actual project, and teams that treat it as a footnote to picking a model tend to burn a whole quarter finding that out. Scope it first, own it. Do that and the model becomes the easy part — genuinely.