An internal AI platform team is a small central group whose product is other teams' productivity with AI: shared model access, guardrails, evaluation tooling and reusable patterns. Build it around a charter of 'make the safe path the easy path', and staff it with two or three senior builders before any managers.
The charter: paved road, not toll booth
The failure mode is a platform team that reviews and approves everything, becoming the bottleneck it was meant to remove. The winning charter is the opposite: build the paved road (model gateway, logging, evals, templates) so the compliant way is also the fastest way.
- One gateway for model access, with logging and cost attribution built in.
- Shared evaluation tooling any team can point at its own workflow.
- Templates for the common patterns: RAG, extraction, triage, drafting.
- Guardrails as defaults, not as review meetings.
The first roles, in order
| Order | Role | Owns |
|---|---|---|
| 1 | Senior AI engineer | Gateway, eval harness, first reference workflows |
| 2 | Platform engineer | Infra, access control, logging, cost visibility |
| 3 | Product manager (part-time at first) | Intake, prioritization, adoption metrics |
| 4 | Enablement / solutions engineer | Embedding with business teams, training, patterns |
How to measure the team
- Adoption: number of teams shipping on the platform.
- Time-to-first-workflow for a new team (target: under two weeks).
- Share of AI usage flowing through the gateway (visibility proxy).
- Cost per workflow trending down as patterns are reused.
