Building an Agentic AI Team: The Roles You Actually Need

Agentic systems, meaning systems where a model plans and executes multi step work using tools, fail in ways that conventional software does not. They fail partially, they fail plausibly, and they fail in ways that are difficult to reproduce. A team staffed as though this were ordinary application development will ship something impressive in a demonstration and unmanageable in production.

The distinguishing property is that the system chooses its own path. A conventional program follows a path you specified and fails visibly when it cannot. An agent selects among options, and when it selects badly it usually produces something that looks like a reasonable answer. There is no exception thrown and no alert raised.

This changes what the team needs. The roles below reflect where the actual difficulty sits, which is in evaluation, containment and operations rather than in prompting or orchestration.

Why agentic systems need different roles

Three properties drive the difference. First, the system chooses its own path, so the space of possible behaviours is far larger than anything you can enumerate in a test suite. Second, failures are plausible: a wrong answer arrives with the same confidence and formatting as a right one. Third, the system takes actions in the world through tools, which means a failure can have consequences beyond an incorrect output.

Together these mean the difficulty is not in getting the system to work. Getting an agent to work in a demonstration is comparatively easy and getting steadily easier. The difficulty is knowing whether it is working, containing it when it is not, and operating it when nobody can reproduce what it did.

The roles

For a team of eight to fifteen owning an agentic capability in production, the following composition works. The proportions are more important than the exact counts.

Evaluation is the hard problem

For conventional software, correctness is usually decidable: the function returns the right value or it does not. For agentic systems, the same task can be completed by many acceptable paths and failure is often a matter of degree. This makes evaluation a genuine engineering discipline rather than a testing activity.

An evaluation engineer builds the datasets, defines what acceptable behaviour means in terms specific enough to measure, designs live monitoring that catches degradation before users report it, and maintains a taxonomy of failure modes that grows as new ones are discovered. That last item is the most valuable artefact the team produces and the one that never gets planned.

Teams without this role do not stop evaluating. They evaluate implicitly, using whatever is easiest to measure, and then optimise it faithfully while the system gets worse at what actually matters.

Tool engineering is most of the work

An agent is bounded by the tools it can use. Most agentic failures that appear to be reasoning failures are actually tool failures: a tool with an ambiguous contract, an error message the model cannot act on, a default that is unsafe when chosen wrongly, or a tool that succeeds partially and reports success.

This is ordinary software engineering, done with more care about interfaces than usual. It does not require machine learning depth, which is useful because it means the largest part of the team can be staffed from a much less constrained talent pool.

The design principle that matters most is that tools should make wrong actions difficult rather than relying on the model to avoid them. A tool that can delete without confirmation will eventually delete, and the correct response is to change the tool rather than to improve the instructions.

• Tool contracts are explicit about what they do and what they will refuse
• Errors are actionable, telling the caller what to do differently
• Destructive actions require confirmation or are unavailable by default
• Partial success is reported as partial, never as success
• Every tool call is traced with inputs and outputs for replay
• Permissions are scoped per tool rather than granted to the agent as a whole

Containment and operations

Containment answers what the system may do without a human. It is an engineering problem rather than a policy problem: permissions, rate limits, spend caps, reversibility and approval boundaries all have to be built.

Operations is the part most often deferred and least deferrable. Agentic systems need tracing detailed enough to reconstruct a decision path, replay so a failure can be reproduced, cost monitoring because spend can escalate quickly and quietly, and an incident process that accounts for the fact that the system may have taken actions rather than merely returned a wrong answer.

This is why operations should sit inside the team rather than being handed to a separate function. Debugging an agentic failure requires understanding why the system chose what it chose, and that understanding lives with the people who built it.

Hiring sequence

The order matters, and the most common error is hiring orchestration capacity before evaluation ownership. That produces a team that can build agents quickly and cannot tell whether any of them work.

Failure patterns in team design

Each of these produces a team that demonstrates well and cannot operate what it built.

• No evaluation owner, so the easiest measurable proxy becomes the objective by default
• Staffing mostly prompt and orchestration work, which is the smallest part of the effort
• Treating tool engineering as plumbing rather than as where most failures originate
• Containment written as a policy document with no engineering behind it
• Operations handed to a separate team that cannot reconstruct why the agent chose what it chose
• No named owner for what good means in the domain

Frequently asked questions