On-Premise AI Observability: Monitor Avatars Without Exporting Customer Content

You can monitor an on-premise AI avatar without sending customer conversations to an external vendor. The practical answer is to collect operational signals by default, keep telemetry inside the customer boundary and treat transcripts, recordings and prompts as a separate, tightly governed evidence class.

That distinction matters. A bank, government department or regulated enterprise still needs to know whether its avatar is available, responsive, safe and correctly configured. “Private” cannot mean invisible to the people operating it. Equally, a dashboard should not become an accidental second copy of every conversation.

The goal is privacy-preserving observability: enough information to detect failures, investigate incidents and improve the service, without routinely exporting the content the system was deployed privately to protect.

Start by separating telemetry from conversation content

Teams often use “logs” as if it describes one harmless data type. In a real-time avatar pipeline, it can mean anything from GPU temperature to a verbatim transcript of a customer discussing their finances.

A workable design separates five planes:

1. Infrastructure telemetry: GPU utilisation and memory, CPU, RAM, disk, network health, container status, temperature, power and node availability.
2. Application telemetry: request counts, queue depth, errors, retries, component latency, dropped sessions and time to first audio or rendered frame.
3. Quality and safety signals: aggregated outcome codes, grounding results, fallback rates, handovers, policy decisions and user-rated success where consent and purpose allow.
4. Security and audit events: authentication failures, privileged access, configuration changes, model or prompt versions, knowledge-base updates and administrative actions.
5. Conversation evidence: audio, video, transcripts, prompts, retrieved passages and generated responses.

The first four can usually provide strong operational visibility without revealing what a person said. The fifth may sometimes be needed for a specific investigation, safety review or quality test, but it should not be switched on merely because a logging library makes it convenient.

This is consistent with the UK Information Commissioner’s Office principle that personal data should be adequate, relevant and limited to what is necessary. It also reflects the UK National Cyber Security Centre’s guidance to log and monitor AI systems after deployment while considering privacy and data-protection requirements.

What should remain inside the customer environment?

For a customer-hosted deployment, the observability stack can sit beside the avatar workload on the customer’s network. Metrics, logs and traces flow to local collectors, then to local storage and dashboards controlled by the customer.

A reference pattern looks like this:

• the browser, kiosk or application starts an interaction using a random correlation identifier;
• speech recognition, retrieval, language-model, text-to-speech, rendering and streaming components attach timings and outcome codes to that identifier;
• local collectors remove or transform disallowed attributes before storage;
• metrics feed the customer’s monitoring and alerting platform;
• audit events feed its security monitoring environment;
• any approved conversation evidence goes to a separate, restricted store with its own retention and access policy;
• support teams outside the boundary receive only an explicitly approved diagnostic bundle, not an automatic stream of customer content.

The correlation identifier is important. It lets an operator see that a session experienced a slow retrieval step, two text-to-speech retries and a WebRTC disconnect without knowing the user’s name or reading the conversation.

OpenTelemetry, for example, documents processors that can filter, redact or transform sensitive attributes before telemetry leaves a collector. NVIDIA’s Data Centre GPU Manager exporter can expose GPU health and utilisation metrics for monitoring systems. These are useful building blocks, not a complete architecture or a guarantee that every default is privacy-safe. Field names, exporters, diagnostic modes and third-party agents still need review.

The metrics that make an avatar operable

A useful dashboard follows the experience through the whole pipeline. GPU utilisation alone cannot tell you that a user waited too long for the avatar to speak.

User-experience signals

• session start success rate;
• median and tail time to first audio;
• median and tail time to first rendered frame;
• turn response latency;
• frame rate, audio-video synchronisation errors and dropped frames;
• microphone permission, device and browser failure categories;
• unexpected disconnect and reconnect rates;
• handover, fallback and task-completion outcomes.

Tail latency matters. An acceptable average can hide a small but important group of very poor sessions, so architecture teams should define percentiles and thresholds against the actual service journey.

Pipeline signals

Measure each stage separately: end-of-speech detection, speech recognition, retrieval, language-model generation, text-to-speech, animation, encoding and network delivery. Record component and configuration versions alongside the timing. When performance changes after an update, operators can then compare like with like.

Keep dimensions bounded. Language, deployment zone, device class, browser family and coarse outcome code can be useful. Customer names, free-text prompts and unbounded document identifiers can create privacy risk and make metrics systems expensive or unstable.

Infrastructure signals

Track GPU memory, utilisation, throttling, temperature, health errors and workload saturation alongside CPU, memory, disk and network signals. Queue depth and admission-control decisions are especially valuable: they show whether the system is approaching its designed concurrency before users experience failure.

For a more detailed approach to load testing and headroom, see Yepic’s guide to GPU sizing for real-time AI avatars.

Safety, security and audit signals

Log the decision, not necessarily the sensitive material that caused it. A policy event can record “restricted topic fallback triggered”, the policy version and the route taken, without duplicating the user’s words into a general-purpose dashboard.

Security teams will also need a durable record of who changed the system, what changed and when. This should cover identities and permissions, model versions, system instructions, knowledge sources, safety policies, deployment artefacts and observability configuration itself. Monitoring that can be silently disabled is weak evidence.

A content-free default is not a content-never rule

There are cases where engineers cannot understand a failure from timing and outcome codes alone. A repeated recognition error in Arabic, an unsafe answer or an intermittent rendering defect may require a sample of the underlying interaction.

The answer is a controlled diagnostic mode, not indiscriminate permanent recording. A defensible process defines:

• the incident or test purpose;
• the smallest required content type;
• the approved users and access route;
• whether masking, sampling or synthetic reproduction is sufficient;
• the start and automatic expiry time;
• the local encrypted destination;
• the deletion and audit evidence;
• the approval required before anything crosses the customer boundary.

For some services, recording a conversation may be inappropriate even during diagnosis. Synthetic test sessions, structured user-reported incident codes or supervised reproduction may be better. The decision depends on purpose, law, sector policy and the risks to the people using the service.

Restricted networks add another consideration: telemetry dependencies must work without relying on public endpoints. Yepic’s restricted-network design guide explains how to inventory hidden online dependencies before describing a system as offline or air-gapped.

Set retention by data class, not by convenience

One retention period for everything is rarely sensible. High-volume infrastructure metrics may only need fine detail for a short operational window, followed by lower-resolution aggregates. Security audit records may have a longer policy requirement. Approved conversation evidence should normally have the narrowest purpose and shortest practical life.

Define the schedule before launch and test deletion. Include backups, replicas, caches and exported incident bundles. A dashboard setting that hides old data is not the same as erasure.

Access should also follow data class. Platform engineers may need component timings but not transcripts. Security teams may need administrative audit events but not model-quality samples. A small authorised group may access quarantined evidence under a recorded break-glass process. The separation reduces both exposure and the number of people placed in an unnecessarily sensitive role.

What can leave the boundary for vendor support?

On-premise does not have to mean unsupported. It means the export path should be deliberate.

A privacy-preserving support bundle might include:

• software, model and configuration version identifiers;
• timestamps converted to an agreed time zone;
• anonymised correlation identifiers;
• stage timings, error codes and retry counts;
• GPU and system health for the relevant window;
• network and browser categories;
• a customer-written description of the observed behaviour;
• the redaction and approval record for the bundle.

The support contract should state who prepares the bundle, who approves it, how it is transferred, where it is stored and when it is deleted. It should also cover remote access, emergency access and the difference between telemetry required for the service and optional product analytics.

Lessons from a real enterprise avatar integration

Yepic’s work with Abu Dhabi Aviation and Oracle illustrates why observability must cover more than the AI model. The delivery involved development and production environments, API and iframe integration, real-time streaming, captions and microphone behaviour, WebRTC and network testing, browser remediation, cybersecurity support and ongoing maintenance.

That project is evidence of enterprise integration and operational testing. It should not be misrepresented as a completed customer-hosted or content-free monitoring deployment. The useful lesson is broader: a real-time avatar is an end-to-end service. When something feels slow or fails, the cause may be a permission, browser, network, streaming or integration issue rather than the generative component.

The same principle shapes Yepic’s private-GPU reference architecture for banks: ownership boundaries, integrations and operational responsibilities need to be designed alongside inference.

Ten acceptance questions for an architecture review

1. Which metrics, logs and traces are collected in normal operation?
2. Can any field contain audio, video, transcripts, prompts, retrieved content, personal identifiers or secrets?
3. Where is each telemetry class processed, stored, backed up and deleted?
4. Can collectors enforce an allowlist and redact disallowed attributes before export?
5. Can a single interaction be diagnosed across every pipeline stage using a non-identifying correlation ID?
6. Are software, model, prompt, policy and knowledge versions recorded for every deployment?
7. Which dashboards and alerts prove user experience, not just server health?
8. How is diagnostic content capture authorised, time-limited, audited and automatically disabled?
9. What information can vendor support receive, and what customer approval is required?
10. Has the team tested monitoring, alerting, deletion and incident-bundle production under realistic load?

Build privacy into the operating model

The best time to decide what an avatar records is before an incident, not during one. Define telemetry fields, retention classes, alert thresholds, access roles and support exports as part of the architecture. Test them alongside latency, failover and capacity.

For banks, governments and regulated enterprises, this creates a stronger position than either extreme. The system is not a black box that nobody can operate, and it is not a surveillance archive created in the name of reliability. It is an observable service with explicit limits.

Yepic supports cloud, private-cloud and customer-hosted avatar architectures through properly scoped implementations. The right model depends on data sensitivity, operational capability, latency, integration and economics. Wherever the workload runs, the operating principle remains useful: measure the service continuously, expose customer content only when there is a defined and governed reason.