A team that estimates a typical feature at 5 story points and delivers it in one sprint, consistently, most of the time, will estimate an AI feature at what looks like a comparable size and miss the sprint by a wide margin, not occasionally but as a pattern. This isn't a team executing poorly, it's a planning method being applied to a kind of work it was never built to estimate. Story points assume the estimator has a reasonably good model of what's involved once the ticket is understood; AI work routinely violates that assumption in ways that are structural, not incidental, and a planning approach that doesn't account for that will keep producing commitments the team can't actually hit.
Why AI work carries a fundamentally different kind of uncertainty
Feature work uncertainty is mostly about implementation detail: how many edge cases does the UI need to handle, how many systems does this integration touch, how much refactoring does this require. That uncertainty is usually resolvable by someone experienced reading the ticket carefully, and it scales roughly with the visible size of the task. AI work has an additional layer of uncertainty that doesn't behave the same way: whether the available data is actually good enough for the task before you've built anything to find out, whether the model behaves reliably on the specific distribution of real inputs it will see rather than the clean examples used to prototype it, and how many iteration cycles an eval-driven improvement loop will need before quality is acceptable, which is inherently open-ended because you don't know the answer until you've run the loop and looked at the results. None of these three unknowns are things a careful reading of the ticket resolves in advance, which is exactly why they don't fit into a story-point estimate the way implementation complexity does.
The assumption story points make, and where it breaks
Story points work reasonably well for feature work because they encode an implicit assumption: uncertainty scales with visible task size, a bigger-looking task has proportionally more unknowns, and an experienced estimator's gut sense of size is a decent proxy for effort. AI work breaks that proportionality specifically. A task that looks small on a ticket, 'improve the model's accuracy on this category of query', can hide fully open-ended uncertainty, it might take one prompt tweak or it might take three weeks of data collection and retraining, and there's no way to tell which from reading the ticket. Conversely, a task that looks large, 'build a full evaluation pipeline for this feature', can be mechanically well-understood and estimate reliably once the eval criteria are defined, because building known infrastructure is implementation work, not exploration. Story points fail here not because the team is bad at estimating, but because the method assumes a correlation between size and uncertainty that simply doesn't hold for a meaningful share of AI work.
The core distinction: explore work vs. ship work
The planning approach that actually holds up starts by classifying work into two categories before estimation happens at all, rather than trying to force every ticket through the same estimation process. Explore work is anything where the answer to 'how good can this get, and how long will that take' is genuinely unknown until you've done some of it: assessing whether existing data supports a new model behavior, testing whether a prompting approach reaches acceptable accuracy, running an initial round of an eval-driven improvement loop to see how much movement the first iteration produces. Ship work is everything that remains genuinely well-understood once the explore phase has answered its question: building the production pipeline around an approach that's already been validated, integrating a model that's already hit its accuracy bar, building the UI around a capability whose behavior is now known. Only ship work should go into a sprint as a committed, story-pointed deliverable; explore work gets planned differently, as time-boxed investigation with an information deliverable, not a feature deliverable.
| Dimension | Explore work | Ship work |
|---|---|---|
| What it produces | Information: is this feasible, how good can it get, what does it need | A deliverable: a shipped feature, integration, or pipeline |
| How it's planned | Time-boxed investigation with a fixed deadline, not a fixed scope | Story-pointed and committed to a sprint, same as regular feature work |
| What 'done' means | The question is answered, even if the answer is 'this doesn't work yet' | The deliverable is shipped and meets its defined acceptance criteria |
| Estimation approach | Estimate the investigation's time-box, not the outcome | Estimate normally; the uncertainty that made this hard has already been resolved |
| Example | Test whether prompting alone reaches 90% accuracy on this task | Build the production pipeline once the prompting approach is validated |
Spike-then-commit as the operating mechanism
The mechanism that makes the explore/ship split actually work day to day is a disciplined spike-then-commit pattern: before any AI-touching work enters a sprint as a committed deliverable, it goes through a short, strictly time-boxed spike, typically a few days, sized to the specific uncertainty, whose only job is to answer the open question well enough to make a real estimate possible. The spike has a hard deadline, not a hard scope, and it ends with one of three concrete outcomes: the approach clearly works and the remaining work can now be estimated normally as ship work, the approach clearly doesn't work and the team needs to try a different approach (which is itself valuable information, not a failure), or the picture is still unclear and the team makes an explicit decision to spend one more time-boxed spike rather than silently letting exploration bleed into what was supposed to be committed sprint work. The discipline that matters most here is refusing to let a spike quietly become the actual work without ever being re-estimated once its question is answered; that silent transition is exactly how sprints keep slipping.
- Time-box every spike to a fixed, short duration, matched to how uncertain the question actually is, not to how much time is available in the sprint.
- Define the spike's success criteria as an answer to a specific question, not as a working feature.
- Re-estimate as ship work immediately once a spike resolves its question, rather than letting the same ticket silently continue as unestimated work.
- Treat 'this approach doesn't work' as a successful spike outcome, not a failure, since it's exactly the information the spike was meant to produce.
- Track how often spikes need a second round; a pattern of frequent second rounds on a certain kind of task is useful data for sizing future spikes better.
Uncertainty-banded estimates for the parts that still need a number
Even ship work in an AI system carries somewhat more residual uncertainty than typical feature work, because model behavior and data quality can still surprise the team during integration even after a spike has validated the core approach. The practical adjustment is an uncertainty band rather than a single point estimate: instead of committing to '5 points', the team commits to a range ('this is a 5, could realistically run to an 8 if the integration surfaces a data quality issue we didn't catch in the spike') and, critically, communicates that range up front rather than discovering it mid-sprint. This isn't a hedge to avoid accountability, it's an honest reflection of where AI work's residual uncertainty actually lives even after the explore phase has done its job, and teams that adopt it consistently find their sprint commitments become more trustworthy, not less, because the band was calibrated in advance rather than discovered as a surprise slip.
Communicating this to non-technical stakeholders
The hardest part of this shift is rarely the planning mechanics, it's getting stakeholders used to normal sprint predictability to accept that some AI work genuinely can't be estimated the way a UI feature can. The framing that lands best isn't 'AI is unpredictable', which sounds like an excuse, it's a distinction most stakeholders already understand from other contexts: research versus execution. Nobody expects a fixed, confident timeline for 'will this experimental drug work', but everyone expects a fixed timeline for 'manufacture and ship the drug once the trial succeeds.' The same framing applied to AI work, this is a short, time-boxed research question, here's exactly when we'll have an answer, and here's what happens next depending on that answer, gives stakeholders a concrete commitment (the spike's deadline and its possible outcomes) even when the downstream work can't be committed yet. The version of this that fails is treating exploration as an invisible, open-ended blob inside a sprint that was presented as a normal committed deliverable; the version that works makes the explore phase itself a visible, time-boxed, reportable unit of work with its own deadline and its own concrete deliverable, an answer.