AI Service-Level Agreements: Measure the Whole Avatar Journey

An AI service-level agreement for a real-time avatar should measure whether people can complete a usable interaction—not merely whether an API returned a response. Availability, tail latency, speech quality, grounded answers, safe failure, media delivery and human handover all belong in the service definition. The agreement must also say where measurement happens and who owns each failure domain.

This matters because an avatar can be technically “up” while a customer hears no audio, waits through an unnatural pause, receives an unsupported answer or cannot reach a person. A single monthly uptime figure hides those failures. For a bank, government department or regulated enterprise, the useful question is: what proportion of eligible interactions completed the required journey within agreed quality and safety limits?

Separate the SLI, SLO and SLA

Teams often use these terms interchangeably, which makes contracts difficult to test.

• A service-level indicator (SLI) is a measured signal, such as session-start success, p95 time to first audible response or completed handover rate.
• A service-level objective (SLO) is the target for that signal over a stated window and workload, such as a required proportion of eligible sessions starting successfully in a calendar month.
• A service-level agreement (SLA) turns selected objectives into commitments and defines what happens when they are missed: notification, remediation, service credits, termination rights or another agreed consequence.

Google's Site Reliability Engineering guidance on service levels makes the same distinction and recommends user-focused indicators. It also cautions that averages can hide poor tail performance. For conversational systems, a median response time can look healthy while the slowest five per cent of turns feel broken. Specify percentiles and distributions, not just means.

Draw the service boundary before choosing targets

No responsible supplier can promise an end-to-end outcome without a defined boundary. Start with the journey, deployment model and workload envelope.

For each service, record the approved channels, locations, languages, client devices, browsers, authentication routes, knowledge sources, integrations and peak-concurrency assumptions. Separate anonymous information journeys from authenticated transactions. State whether the target applies to production traffic, synthetic probes, business hours, a 24-hour service or a particular site.

Then place every component on an ownership map:

1. session admission, identity and access;
2. microphone, camera and client permissions;
3. speech capture, turn detection and recognition;
4. retrieval, reasoning, policy checks and tool calls;
5. text-to-speech, voice and avatar rendering;
6. encoding, WebRTC transport and client playback;
7. fallback, human handover and incident recovery.

A public-cloud service places more operational responsibility with the provider. A private-cloud design may split it across the provider, cloud platform and customer. In a customer-hosted implementation, the organisation normally controls more of the GPU estate, network, orchestration, identity and monitoring. That control can support sovereignty and integration requirements, but it also moves more operating responsibility inside the customer's boundary.

“On-premise SLA” should therefore not mean that a vendor guarantees customer power, capacity or network paths it neither operates nor observes. The contract can instead define software-support commitments, diagnostic evidence, defect response, tested releases and escalation, alongside customer obligations for the agreed hardware, capacity, access windows and infrastructure health.

Use eight SLI families for the avatar journey

1. Session availability

Measure the proportion of eligible attempts that establish the intended interaction. Define the denominator carefully: are denied identities, unsupported browsers, planned closures or traffic beyond the contracted workload included? Record the measurement point at the user edge as well as the application gateway. Server health is only a proxy if the browser cannot start a session.

2. Turn completion

A successful HTTP request is not a successful conversational turn. Define success as the user being heard, the system producing an allowed response and the client playing it to completion—or entering an approved fallback. Track abandoned, timed-out, duplicated and silently dropped turns. This indicator catches failures between components that remain individually “available”.

3. End-to-end latency

Measure from a user-observable start point to a user-observable result: end of speech to first audible response, interruption to stopped playback, or tool confirmation to spoken acknowledgement. Break that budget into speech recognition, retrieval, model generation, tool calls, speech generation, rendering and delivery, but make the end-to-end measure decisive.

Report p50, p95 and, where volume supports it, p99 by relevant workload class. A long authenticated transaction and a short kiosk enquiry should not share one blended target. Yepic's GPU capacity-planning guide for real-time avatars explains how to validate these thresholds under realistic concurrency rather than extrapolating from a demonstration.

4. Media quality

A response may arrive on time but be unintelligible or visually frozen. Track audio concealment, packet loss, jitter, reconnects, video freezes, delivered frame rate and audio-video synchronisation at the client. The W3C WebRTC statistics specification defines many useful browser-side signals, while also warning that statistics can create privacy considerations. Choose the minimum fields needed and govern their retention.

5. Task quality and correctness

Operational availability does not prove that an answer is correct. Maintain a governed evaluation set for each important task and language. Measures might include speech-recognition accuracy on an accepted corpus, answer grounding, citation validity, transaction confirmation, tool-call success and refusal behaviour. Version the knowledge, prompts, policies, models and test set so that a score can be reproduced.

Do not turn these measures into a blanket guarantee of truth. Retrieval can improve grounding but cannot eliminate hallucination. Legal, financial or clinical outcomes require their own controls, approvals and human boundaries.

6. Safety and handover

Specify whether restricted requests are refused, redirected or escalated, and measure whether that route completes. A handover is not successful because the avatar displayed a button; the required context must reach the authorised human channel, within the defined time, without exposing unnecessary data. False refusals also matter: a system that blocks legitimate users can meet a narrow safety target while failing its service purpose.

7. Capacity and overload behaviour

Targets are meaningful only inside a declared workload envelope. Include concurrent sessions, arrival bursts, language mix, response length, avatar resolution, frame rate and failure reserve. State the overload policy: admission control, honest waiting state, reduced visual quality, audio-only mode, text and captions, an approved cloud route or human service.

Graceful degradation must be task-specific. Audio-only may be an acceptable available state for an information service but not for an accessibility requirement that depends on visual content. The fallback should never disguise failure by silently providing a different service.

8. Support and recovery

Define severity levels, acknowledgement times, update cadence, restoration objectives, escalation routes and evidence required from each party. Separate response time from restoration time. State how security fixes, model changes and configuration updates are tested and released. Link these commitments to the avatar disaster-recovery and failover plan, rather than treating incident support as a substitute for resilience.

Write every objective as a testable record

A useful SLO record fits on one line but answers nine questions:

1. What user journey and workload class does it cover?
2. What is a good event?
3. What is the denominator?
4. Where is the signal measured?
5. What percentile or proportion is required?
6. Over what window is it calculated?
7. What exclusions and maintenance rules apply?
8. Which evidence source is authoritative?
9. Who owns detection, diagnosis and recovery?

For example, “99.9% uptime” leaves nearly all of these unanswered. A stronger definition is: eligible authenticated sessions, during agreed service hours and within the tested concurrency envelope, that reach first audible response and stable playback without an unexpected disconnect, measured by approved client probes and production outcome codes over a calendar month. The percentage still needs negotiation; the point is that the event can be reproduced and disputed evidence can be reconciled.

Do not multiply component SLAs and call the result journey availability unless the dependencies and failure relationships justify it. Nor should a provider exclude every upstream system until the commitment becomes meaningless. Use one user-journey indicator, supported by component indicators that locate responsibility.

Add an error budget and degradation ladder

An objective below 100% creates an error budget: the amount of allowed failure in the measurement window. That budget should govern decisions. Rapid consumption can trigger a change freeze, capacity review or release rollback; sustained healthy performance can support controlled improvement. Google's SLO implementation guidance describes how objectives can drive data-informed operational decisions rather than becoming passive reports.

Agree the service states before launch:

1. full voice-and-video interaction;
2. reduced visual quality with clear audio;
3. audio-only interaction;
4. text and captions;
5. human handover or an honest service-unavailable state.

For each state, decide whether it counts as available for each task, how users are informed and how long the service may remain there. Test the transitions in the pilot-to-production stage-gate plan.

Make measurement private by design

An SLA needs evidence, but evidence does not require a permanent archive of conversations. Correlation identifiers, timestamps, stage timings, outcome codes, component versions and bounded client statistics can establish most operational indicators. Keep audio, transcripts, prompts and retrieved content in a separate evidence class with explicit purpose, access and retention.

For customer-hosted systems, metrics can remain in the customer's monitoring environment. External support can receive an approved diagnostic bundle rather than a continuous export. Yepic's guide to private observability for on-premise avatars sets out that separation. The UK National Cyber Security Centre also recommends monitoring system behaviour and inputs after deployment in its secure AI operation and maintenance guidance.

Use enterprise delivery evidence carefully

Yepic's integration with Abu Dhabi Aviation and Oracle shows why an avatar service boundary must extend beyond the model. Delivery included development and production environments, API and iframe integration, real-time streaming, captions and microphone behaviour, WebRTC and network testing, browser remediation, cybersecurity support and ongoing maintenance.

That is evidence of end-to-end enterprise integration work, not a claim that the project used this SLA framework or was customer-hosted. Yepic can support cloud, private-cloud and customer-hosted avatar architectures through properly scoped custom implementations. The specific commitment has to follow the chosen components, operating model, workload and evidence available to both parties.

Twelve questions for procurement

1. Which user journeys are contractually in scope?
2. What is the authoritative definition of an available, completed interaction?
3. Which latency percentiles apply to each workload and language?
4. Where are session, media and quality indicators measured?
5. How are browser, network, identity and knowledge failures attributed?
6. What capacity envelope and failure reserve underpin the targets?
7. Which degraded states count as available for each task?
8. How are semantic quality, grounding and safe handover tested separately from uptime?
9. What evidence remains inside the customer environment, and what may be shared?
10. Who owns monitoring, triage, restoration and stakeholder communication?
11. What happens when releases consume the error budget or change performance?
12. What remedies apply, and do they support recovery rather than merely issue credits?

Turn the pilot into a measurable service

The best time to write an AI service-level agreement is before a successful demonstration becomes a production dependency. Use the pilot to establish the workload envelope, measurement points, acceptance corpus, failure modes and shared-responsibility matrix. Then negotiate targets from observed evidence.

There is no universal uptime, latency or support figure that is credible for every real-time avatar. The right SLA is a concise description of the experience that must work, the conditions in which it must work, the evidence that proves it and the people accountable when it does not.