How to Structure an AI Innovation Team Inside a Nano GCC
Small applied artificial intelligence teams fail in a consistent pattern. They are staffed almost entirely with machine learning specialists, they underinvest in data engineering, nobody owns the definition of what good means, and they hire the supporting layer before the senior core. The result is a team that can build models and cannot get one into production and keep it there.
The structural problem is that AI work is usually planned as a modelling exercise when it is mostly a data and systems exercise. Most production machine learning failures are data reliability failures, evaluation failures or operational failures, and none of those are solved by more modelling capacity.
This guide sets out the roles a small applied AI team actually needs, the ratios that work at that scale, the hiring sequence, and the one role that is missing from almost every plan.
Key points
- Most production AI problems are data and evaluation problems, not modelling problems
- A small team needs fewer machine learning specialists than assumed and more data engineers
- Evaluation ownership is the role most commonly missing entirely
- Hire the senior applied lead first; everything else follows from that decision
- The team must own an outcome, or it will produce demonstrations rather than systems
What the work actually consists of
Before deciding roles, it is worth being honest about where the effort goes. In a team taking a machine learning capability into production and operating it, modelling is a minority of the work. Data acquisition and reliability, evaluation design, serving infrastructure, monitoring, cost control and failure analysis consume most of it.
This is not a criticism of modelling; it reflects that modelling is the part that current tooling has made dramatically easier, while the surrounding work has not become easier at the same rate. A team staffed as though modelling were the bulk of the effort will be over specialised in the part that is now cheapest.
Approximate distribution of effort in production AI work
Illustrative distribution for planning rather than measured research. The shape varies by domain, but modelling is a minority of the effort in almost every production context.
The roles a small team actually needs
For a team of eight to fifteen people owning a production AI capability, the following composition works. Note how few pure machine learning specialists it contains.
One person, hired first. Owns architecture, evaluation strategy and the hard calls about what is and is not working. This is the constrained hire and the plan should be built around their availability.
One to three, depending on scope. People who have taken systems into production and operated them. Their value is knowing what fails, which is knowledge that cannot be trained quickly.
Typically the largest group and consistently under resourced. Pipelines, feature availability, data quality monitoring and lineage. Most production failures originate here.
Strong software generalists owning inference serving, latency, observability and cost. No machine learning depth required, and readily available.
The role most often missing. Owns what good means for this system, how it is measured, and when it is good enough. Without it, the team optimises a proxy nobody agreed to.
Part time or embedded. Someone who understands the business problem well enough to distinguish a technically impressive result from a useful one.
The role that is always missing
Evaluation ownership is absent from most AI team plans, and its absence is the most reliable predictor that a team will produce systems that perform well on a benchmark and badly in use.
The role owns three questions. What does good mean for this system in business terms rather than metric terms? How do we measure that, including the failure modes that matter most? And when is it good enough to ship, which is a judgement rather than a threshold.
Without a named owner, evaluation defaults to whatever metric is easiest to compute, and the team optimises it faithfully. The system then improves on that metric while getting worse at the thing the business actually cared about, and nobody notices until it reaches users.
Ratios at small scale
The composition below is a starting point for a team of ten to twelve owning a production capability. The proportions matter more than the absolute numbers, and the ratio that surprises people is data engineering to machine learning.
| Role | Count in a team of ~12 | Seniority | Notes |
|---|---|---|---|
| Senior applied ML lead | 1 | Lead | Hire first; participates in every subsequent interview |
| Senior applied ML engineer | 2 | Senior | Production experience is the requirement, not research depth |
| Data engineer | 3 to 4 | Mixed, senior weighted | The largest group and the most commonly under resourced |
| Platform and serving engineer | 2 to 3 | Mixed | Strong generalists; no ML background needed |
| Evaluation and product owner | 1 | Senior | Frequently missing entirely; often the highest leverage hire |
| Domain specialist | 0.5 | Senior | Embedded or shared rather than dedicated |
Note the ratio of data engineers to machine learning engineers. Most plans invert it, which produces a team that can model well and cannot supply itself with reliable data.
Hiring sequence
Sequence matters more than speed. Hiring the supporting layer first produces capacity waiting for direction, and direction that arrives after the team is assembled has to work with people it did not choose.
Senior applied ML lead
Everything waits on this hire. They should participate in every subsequent interview, which is what makes the group a team rather than a set of individuals.
Evaluation and product owner
Hire this before the engineers. Building without an agreed definition of good produces work that has to be redone once the definition arrives.
Data engineers
Before the modelling capacity, because reliable data is the precondition for everything else. A modelling team without data engineering spends its time doing data engineering badly.
Senior applied ML engineers
Now the team can supply them with the data and the evaluation criteria they need to be productive from the start.
Platform and serving
Can join in parallel with step 4. This layer is the most available in the market and the least constrained.
Why this suits a small capability centre
A team of ten to fifteen is close to the natural size for a first production AI capability, which makes it a good fit for a small capability centre rather than a compromise. The seniority profile is heavy, which matches how a Nano GCC should be staffed anyway, and the work requires ownership of an outcome, which is what a small centre is designed to provide.
The condition is the same as for any capability centre: the team must own something real. An AI team given a series of proofs of concept to build will produce proofs of concept indefinitely, and after two years will have nothing in production. The mandate has to include operating what it builds, or the team never learns the lessons that make it valuable.
- The team owns a production outcome, not a series of prototypes
- Evaluation criteria are agreed with the business before building starts
- The team operates what it ships, including the incidents
- Data engineering is resourced as the largest group, not as support
- The senior lead was hired first and chose the rest of the team
- Someone in the team can say what good means in business terms
Structural mistakes
These produce a competent team that never reaches production, which is the most expensive possible outcome because it looks like progress throughout.
- Staffing mostly machine learning specialists and treating data engineering as support
- Writing one AI engineer role that requires research depth, production experience and data engineering at once
- Leaving evaluation unowned, so the easiest metric becomes the objective by default
- Hiring the supporting layer before the senior core
- Giving the team proofs of concept rather than a production outcome
- Separating the team from operating what it builds, which removes the feedback that creates judgement
Resource the data layer and name the evaluation owner,These two decisions account for most of the difference between AI teams that reach production and AI teams that produce impressive demonstrations for two years.
Frequently asked questions
How many machine learning specialists do we actually need?
For a first production capability, commonly two to four including the lead. Plans calling for eight or more usually conflate machine learning roles with the data and platform engineering that should support them, which produces an unfillable plan and a badly balanced team.
Why do data engineers outnumber ML engineers?
Because most production machine learning failures are data failures: missing features, stale pipelines, silent quality degradation, lineage that nobody can trace. Modelling has become dramatically easier while data reliability has not, so effort has shifted accordingly.
What exactly does the evaluation owner do?
They define what good means in business terms, design how it is measured including the failure modes that matter, and decide when a system is good enough to ship. It is a product role with quantitative depth rather than a machine learning role, and it is the most commonly missing position.
Can one person cover evaluation and product?
In a team of this size, usually yes, and it is often better because the two questions are closely related. What does not work is leaving both unowned and expecting the engineers to infer the answer, which they will do by choosing whatever is easiest to measure.
Should the team include a researcher?
Only if the roadmap genuinely requires advancing the state of the art, which is rare. For applied production work, a senior engineer who has operated systems is more valuable than a researcher who has not, and considerably easier to hire.
What size should we start with?
Six to eight people is a workable start: the lead, an evaluation owner, two data engineers, one senior applied engineer and one platform engineer. Grow from there once the first capability is in production rather than staffing to target size up front.
Does the team need to operate what it builds?
Yes. Separating building from operating removes the feedback loop that creates production judgement, which is the scarce capability you were trying to acquire. A team that never sees its systems fail will keep building systems that fail in the same ways.
How long before a small AI team reaches production?
With the lead hired first and a real outcome to own, six to nine months for a first production system is realistic. Teams given a series of proofs of concept commonly reach eighteen months with nothing operating, which is a mandate problem rather than a capability problem.
Sources & further reading
- NASSCOM — https://nasscom.in/
- Stanford HAI — https://hai.stanford.edu/
- McKinsey & Company — https://www.mckinsey.com/
- Deloitte — https://www.deloitte.com/
Build the AI team in the right order
Hexominds builds applied AI teams in India with the senior core first, the evaluation ownership named, and the data layer resourced properly.