AI job titles are notoriously fluid. “AI engineer” can describe people working on model integration, data pipelines, backend systems, or evaluation. Building a team around titles alone can leave you with several people who know how to fine-tune a model but no one who can reliably get the right data out of the ERP.
A practical AI project team structure starts by separating the work into distinct responsibilities. Six roles cover most of the core functions, although smaller teams can combine some of them.
The Six Roles Every AI Project Team Needs
Data Engineer
The data engineer builds and maintains the pipeline between source systems and the systems that consume the data, whether that is a retrieval layer, feature store, or application database.
The role typically includes handling schema changes, resolving entities across systems, capturing permission metadata, managing data quality, and defining reindexing or synchronization strategies.
Data engineering often becomes one of the largest areas of effort during the early stages of an enterprise AI project. If the underlying data is inaccessible, inconsistent, or poorly governed, the model layer cannot compensate for it.
Applied AI Engineer
The applied AI engineer owns the model-facing part of the system: retrieval design, prompt and context engineering, model selection and routing, and fine-tuning when it is actually justified.
This is the role most people associate with an AI project, but it is only one part of the overall system. Depending on the use case, applied AI work may represent roughly 20% to 30% of total project effort.
Backend Engineer
The backend engineer builds the software surrounding the model.
That includes APIs, system integrations, authentication, authorization, permission filtering, queues, caching, data access, and the application logic required to turn a model capability into a production service.
These are conventional software engineering responsibilities, but they are critical. A technically strong model does not become a production system without the surrounding application infrastructure.
Evaluation Lead
The evaluation lead owns how system quality is measured.
This includes building and maintaining the evaluation harness, defining grading methodologies, coordinating domain experts, managing test datasets, and running regression evaluations when prompts, models, retrieval logic, or other system components change.
On a small project, the applied AI engineer may also handle evaluation. What matters is that the responsibility is explicitly assigned rather than treated as something the team will “get to later.”
Domain Expert
The domain expert usually comes from the client organization rather than the delivery team.
This person defines what a correct answer or acceptable outcome actually looks like, reviews evaluation cases, identifies important edge cases, and helps the technical team distinguish real business requirements from assumptions.
The role also requires actual time allocation. Depending on the project stage, one to two days per week during design and evaluation can be a reasonable starting point.
Without consistent domain input, a system can perform well against technical metrics while still failing to solve the problem the business actually cares about.
Delivery Lead
The delivery lead manages scope, coordinates decisions, tracks dependencies, and keeps the project moving through its decision gates.
This role can be particularly important in AI projects because some of the most consequential decisions involve uncertainty. After data assessment, for example, someone needs to be willing to recommend changing scope, revisiting assumptions, or stopping the project altogether if the evidence does not support continuing.
How the Ratios Shift by Phase
An AI team should not necessarily maintain the same composition throughout the project. Different phases require different skills, and keeping every role fully staffed from start to finish can create unnecessary cost.
Discovery and Data Readiness
Data engineering tends to take the largest share of the technical effort during this stage.
The domain expert and delivery lead should also be actively involved, while the applied AI engineer contributes to feasibility assessment without necessarily working full time on the project.
The goal is to establish whether the data, use case, and technical constraints support the proposed system before committing to the full build.
Architecture and Evaluation Design
The evaluation lead and domain expert become more involved here.
This is where the team defines what “good” means, establishes the evaluation dataset and methodology, and translates business requirements into measurable acceptance criteria.
Compressing this stage can create problems later. If the team has no agreed method for evaluating quality, discussions about whether the system is “good enough” eventually become subjective.
Build
Backend and applied AI engineering typically account for most of the effort during implementation.
Data engineering remains involved but at a lower level, primarily supporting data issues, pipeline changes, and integration requirements. The domain expert can usually move to a lighter schedule, often a few hours per week for evaluation and review.
Production Deployment
Backend engineering and the people responsible for operating the system become more important as the project moves toward production.
The applied AI engineer’s involvement may decrease substantially unless the deployment uncovers model, retrieval, or performance issues that require further iteration.
Operation
Once the system is stable, the ongoing team can be much smaller.
Operational work may require only a fraction of an engineer’s time, supplemented by periodic attention from the evaluation function for regression testing, quality monitoring, and model or prompt changes.
Firms structured as a full AI software development firm rather than a staffing supplier can often adjust team composition across these phases instead of assigning the same fixed group throughout the engagement. The difference is significant: you are paying for the capabilities required at each stage rather than maintaining a static team regardless of the work.
Realistic Team Sizes
A first enterprise AI project does not necessarily require a large team. Four to six people at peak is a reasonable starting range for many projects, with the exact number depending on scope and integration complexity.
Large teams can create their own inefficiencies during an early project. Much of the work has sequential dependencies, particularly before the data layer is stable, so adding people does not always create more parallel capacity.
A common structure might look like:
- One data engineer
- One applied AI engineer
- One backend engineer
- One delivery lead at approximately 50% allocation
- One evaluation lead at approximately 50% allocation
- One client-side domain expert at approximately 40% allocation
That represents roughly four full-time equivalents at peak.
The team can grow once multiple use cases are being developed on top of shared infrastructure. At that point, the work becomes easier to parallelize and additional engineering capacity can produce more direct output.
The Role Most Teams Are Missing
The evaluation lead is often the easiest role to leave off the org chart.
The title itself is relatively new, and organizations often assume evaluation is simply part of QA or the applied AI engineer’s responsibilities. But evaluation in AI systems has a broader purpose than conventional software testing.
Without a clearly owned evaluation function, several problems tend to appear:
- No representative test set exists.
- The evaluation dataset is created once and then becomes outdated.
- Teams cannot reliably determine whether a prompt or model change improved performance.
- Quality discussions rely heavily on subjective reviews.
- Model migrations become reactive rather than measured.
The role does not necessarily require a dedicated full-time employee. What matters is that someone owns the evaluation methodology, test data, grading process, and regression workflow.
Providers that list evaluation as a distinct deliverable in their AI software development services are making that responsibility explicit. When evaluation is treated only as a subset of generic QA, teams can miss the fact that AI quality requires its own datasets, metrics, and review process.
FAQ
How many people do you need on a first AI project?
For many enterprise projects, four to six people at peak is a practical starting range, representing roughly four full-time equivalents in a common team structure.
The actual requirement depends on the number of systems being integrated, data complexity, security requirements, and the scope of the application.
Do you need a data scientist on an AI development team?
Not necessarily.
For many applied LLM projects, the core team may need a data engineer and an applied AI engineer rather than a traditional data scientist.
A data scientist becomes more relevant when the project involves statistical modeling, predictive analytics, experimentation, or training custom models rather than primarily orchestrating and integrating existing foundation models.
Can one person cover multiple roles on a small AI project?
Yes, as long as the combined responsibilities do not create a bottleneck.
Applied AI engineering and evaluation can often be combined. Data engineering and backend engineering can also overlap in smaller teams.
Combining data engineering with applied AI work requires more caution because both areas can become demanding at the same stage of the project.
What is an evaluation lead, and why does the role matter?
The evaluation lead owns how the quality of an AI system is measured.
That includes the test set, grading rubric, evaluation methodology, regression process, and coordination with domain experts.
Without clear ownership, every change to the system can become a judgment call. With an evaluation process in place, teams can compare versions and make changes based on evidence.
Staffing the Client Side
The delivery team is only half of the staffing equation. The client organization needs its own people involved throughout the project.
At minimum, identify:
- A domain expert with dedicated time for requirements and evaluation.
- A technical owner who will eventually inherit and operate the system.
- A business owner who can make decisions about priorities and acceptable quality levels.
- Data owners for each source system who can approve access and resolve data-related blockers.
The last group is easy to overlook.
A development team can be ready to start while access to an ERP, CRM, document repository, or other source system remains stuck in an internal approval process. That kind of dependency can delay an AI project for weeks without any technical problem in the AI system itself.
Identify the relevant data owners during scoping and start the access process before development begins. It is one of the simplest ways to protect the schedule, and it costs far less than discovering the dependency halfway through the project.
