Delivery manager · Sprint Reply
The hard part was never
the model.
I run AI engagements for large organisations. Most of the work sits either side of the technology — deciding what is actually worth building, and making sure the people funding it and the people building it are describing the same thing.
The same problem, two languages
Pick a problem, then switch the framing.
- The decision
- Whether you are trying to deflect volume or shorten handling time. They are different investments and you cannot lead with both. Deflection touches the customer directly and carries brand risk; handling time is internal and much safer to get wrong on the first pass.
- The risk if we’re wrong
- An assistant that answers confidently and incorrectly on billing or eligibility is a liability you hear about from a regulator rather than a dashboard. The failure mode is silent — a wrong answer still closes the ticket and still looks like success in the numbers.
- How we’d know it worked
- Deflection rate on a fixed ticket mix, cost per contact, and satisfaction on assisted tickets measured against a held-back control group. Not “tickets touched by AI”, which will go up regardless of whether anything improved.
- Where I’d start
- Retrieval over the existing knowledge base and resolved ticket history. No fine-tuning yet — the labels aren’t clean enough and it would bake in today’s answers, including the wrong ones, at exactly the moment the policies are being rewritten.
- How we’d evaluate it
- Roughly three hundred real historical tickets with the resolution that was actually correct, graded by the support leads rather than by the team building it. Run on every change. Track refusal rate separately from accuracy — a system that hedges on everything scores beautifully and helps nobody.
- What to watch for
- The knowledge base is stale and nobody has said so. Retrieval will surface a 2019 policy with total confidence. Content freshness quietly becomes an engineering dependency and it is never in the original estimate.
- The decision
- What “strategy” is standing in for. It is usually one of three questions — where we allocate capital, how we respond to a competitor, or what we will and won’t allow. Those need different documents and different people in the room, and conflating them is why these papers land flat.
- The risk if we’re wrong
- A strategy written as a technology inventory is out of date within a quarter. One written as a set of positions — where we accept risk, where we don’t, who decides — stays useful for years and survives the next model release.
- How we’d know it worked
- Whether it moved money. If the budget looks identical the quarter after it landed, it was a document rather than a strategy, however well it was received in the room.
- Where I’d start
- Honestly, there is no engineering work in the first few weeks, and saying that out loud protects the team from being measured on the wrong thing. The useful output is a capability audit: what data can lawfully be used for what, where it physically lives, and how long access actually takes. That is nearly always the binding constraint, not model choice.
- How we’d evaluate it
- Take two or three concrete candidate use cases and build the thinnest possible version of each. The point is not a demo — it is finding out which integration is going to be the painful one before it is on a plan with a date attached.
- What to watch for
- Being handed a platform shortlist before anyone has agreed what the platform is for. The evaluation criteria then get written backwards from whichever vendor gave the best presentation, and the whole exercise becomes ceremonial.
- The decision
- Whether to productionise it, restate it, or stop. Stopping is a legitimate outcome and it needs saying early and plainly, because the alternative is a pilot that gets quietly re-scoped for two years and consumes a good team.
- The risk if we’re wrong
- Pilots stall for organisational reasons far more often than technical ones — no named owner with it in their operating budget, no route through the change process. Funding a better rebuild does not fix either of those, and you will be back here next year with a more expensive version of the same conversation.
- How we’d know it worked
- Does a named leader have it in their objectives and their budget for next year? If not, it is not going to production whatever its quality, and the honest thing is to say so now.
- Where I’d start
- The distance between a pilot and production is rarely model quality. It is authentication, audit logging, rate limits, the failure path, and somebody on call. That work is unglamorous and routinely three to four times the original effort, which is the number nobody wants in the room.
- How we’d evaluate it
- Before rebuilding anything, run the existing pilot against current real volume and the genuinely awkward cases. The accuracy that impressed everyone was almost certainly measured on a curated set that the builder chose.
- What to watch for
- It was probably built on a personal API key, a spreadsheet extract, and one person’s undocumented understanding of the business rules. Find that person before they change teams.
- The decision
- Whether you are buying a product or funding its development. Both can be perfectly reasonable positions to take — but only one of them should be priced as a product, and only one of them should carry a three-year term.
- The risk if we’re wrong
- Committing against a roadmap rather than a release. Your leverage is at its absolute maximum before signature and effectively zero after it, so anything that matters has to be a contractual milestone rather than a slide.
- How we’d know it worked
- Ask for a reference customer at your scale, in your sector, running it in production today — and ask to speak to them without the vendor on the call. If that cannot be arranged, you have your answer.
- Where I’d start
- Ask what happens when it is wrong. A demo only ever shows the success path. Ask to see the logs from a failed run, the retry behaviour, and precisely how a human gets into the loop — that conversation separates a product from a prototype inside ten minutes.
- How we’d evaluate it
- Your data, your twenty hardest cases, and your team pressing the button. Not their demo set and not their engineer driving. A vendor who won’t allow that is telling you something useful at no cost to you.
- What to watch for
- “Agent” meaning a scripted chain with one model call in it. That is often the right design and I would frequently recommend it — but it should be priced like software, not like autonomy.
01 — What I do
Three places I’m useful
-
Decide
Working out what’s worth building
Most of the value here is in what gets talked out of the plan. I work with sponsors to separate the problems where AI is genuinely the right instrument from the ones where it is an expensive way of avoiding a process fix.
-
Build
Getting it into production
Running delivery alongside the team actually shipping it — scope, sequencing, the integration nobody costed, and the honest conversation about the date. I am not writing the code, but I can tell when an estimate is optimistic and why.
-
Land
Making it stick after launch
A system nobody uses is indistinguishable from one that was never built. Adoption, training, the operating model, and the question of who owns it on the Monday after the project team leaves.
02 — Selected work
Four engagements
Client names withheld. Select an item for the detail.
-
Placeholder content — replace before publishing
The brief
Property claims were being routed by hand out of a shared mailbox. Complex claims sat in the queue behind simple ones, and nobody had a view of what was waiting until they opened it. The ask arrived as “can we use AI on the mailbox”, which is not yet a problem statement.
What I did
Ran discovery with the claims operation and the data team together rather than separately, and scoped it deliberately narrowly: classification and routing only, with no automated decisions touching payment. Built the evaluation set out of historical claims graded by senior handlers before any development started, and used it as the release gate rather than as a report at the end. Managed delivery through to production, including the audit trail the regulator would eventually ask for.
Outcome
[FIGURE NEEDED] reduction in average time-to-route, from [X] to [Y], across roughly [N] claims a month. Handler-graded routing accuracy of [Z]% at go-live, held for [period].
-
Placeholder content — replace before publishing
The brief
Several thousand engineers working from a documentation estate spread across three systems, two of which were only reachable from the corporate network. The answers existed. They were simply not findable by somebody standing in a substation.
What I did
Retrieval over the existing document set, with no fine-tuning — the corpus changed weekly and the labels were not clean enough to justify it. Most of the effort was unglamorous: getting document ownership formally assigned so that content freshness had a name against it, and designing for intermittent connectivity rather than pretending it away. Ran a single depot as the pilot before widening, and made the depot manager a co-owner of the rollout rather than a recipient of it.
Outcome
[FIGURE NEEDED] — e.g. median time-to-answer down from [X] to [Y]; [N] engineers active weekly by month [M]; [Z] escalations avoided per month.
-
Placeholder content — replace before publishing
The brief
Nine teams building independently, three of them unknowingly on the same problem, and no agreed position on what customer data could be used for what. The stated request was a centre of excellence. The actual problem was that nobody knew who was allowed to decide.
What I did
Not an engineering engagement. Mapped what was genuinely in flight — as opposed to what was on slides — ran a data-access and capability audit, and wrote a set of decision rights: what a team may start on its own authority, what needs review, and what is off the table entirely. Then attached it to a funded portfolio so that it changed a budget rather than becoming a document people cite and ignore.
Outcome
[FIGURE NEEDED] — e.g. [N] duplicated initiatives consolidated, [£X] of committed spend redirected, time-to-approval for a new initiative down from [X] to [Y].
-
Placeholder content — replace before publishing
The brief
A vendor proposal for an agent platform claiming end-to-end automation of a back-office process, with a three-year commitment attached and an internal sponsor who had already seen the demo twice and liked it.
What I did
Restructured the evaluation so it ran on the client’s own data and their twenty hardest cases, with the client’s team driving rather than the vendor’s engineer. Separated what shipped today from what sat on the roadmap and insisted the two be priced differently. Wrote the recommendation for a committee that included people who do not buy software for a living, which meant the technical findings had to survive translation without losing their teeth.
Outcome
[FIGURE NEEDED] — e.g. term reduced from [X] to [Y], [£Z] of commitment made contingent on delivered milestones, accuracy on the client’s own case set measured at [N]% against a claimed [M]%.
03 — How I think about it
Four positions
-
i
The bottleneck is rarely the model.
By the time a programme is in trouble, the cause is almost always data access, an unowned process, or a decision nobody has the authority to make.
-
ii
A pilot that can’t name its production owner is a demo.
Ask who has it in their budget next year. A shrug is a complete answer, and it is much cheaper to hear at the start than at the end.
-
iii
Evaluate before you build, not after.
A test set of real cases, graded by the people who actually do the job, is worth more than any amount of model selection or prompt refinement.
-
iv
Estimate the integration, not the intelligence.
The model call is the cheap part. Authentication, audit, and the awkward legacy system are where the quarters actually go.
04 — About
Alexander Withers
I am a delivery manager at Sprint Reply, where I run AI engagements for large enterprise clients. In practice that means sitting between the people who fund the work and the people who build it, and being accountable for the fact that those two groups are describing the same thing.
I came to this from the business side rather than from engineering, but I have spent enough time close to delivery to be useful in a technical conversation — enough to tell whether an estimate is honest, whether a demo is quietly avoiding the hard case, and whether a vendor is describing something that exists today or something they intend to build.
The work I am proudest of is usually the work that got smaller. Scope that came down because we found the real constraint early; a pilot stopped in week six because there was no owner for it; a vendor commitment restructured because the capability was on a roadmap rather than in a release.
I am based in the UK [confirm location] and work mostly with organisations large enough that the interesting problems are organisational as well as technical.
05 — Contact
Get in touch
Happy to talk about delivery, evaluation, or a proposal you are not sure about.
- Email [email address needed]
- LinkedIn [LinkedIn URL needed]
- GitHub [GitHub URL needed]