On-Premise AI Avatars for Banks: A Private-GPU Reference Architecture

An on-premise AI avatar for a bank is not simply a cloud product copied onto a server. It is a complete conversational system whose trust boundaries, model dependencies, identity controls, data flows and operating responsibilities have been deliberately redesigned for the bank’s environment.

The useful question is therefore not “Can the avatar run on our GPUs?” It is: “Which parts of the interaction must remain under our control, which services may cross the boundary, and how will the whole system be secured and operated?”

This reference architecture gives bank technology, security and risk teams a practical starting point. It is not a universal blueprint. A customer-hosted deployment must be scoped around the bank’s use case, infrastructure, policies and chosen speech, language and integration components.

What does on-premise mean for an AI avatar?

A real-time AI avatar is a chain of systems, not a single model. A spoken customer question may pass through audio transport, speech recognition, language detection, retrieval, a language model, policy checks, text-to-speech and visual rendering before the response returns to the screen.

“On-premise” should describe the parts of that chain that execute inside a customer-controlled environment. That may be a bank-owned data centre, a dedicated private region or a sovereign facility operated under the bank’s policies. The label alone does not answer where data is processed.

Before procurement, draw the architecture and mark every external dependency. A deployment is only as private as the service calls it makes during a live conversation.

A private-GPU reference architecture for banks

1. The interaction layer

The customer or employee may meet the avatar through a branch kiosk, authenticated web portal, mobile application, adviser desktop or contact-centre interface. Each channel brings different controls. A public kiosk needs session clearing and physical privacy. A mobile journey needs device and application security. An employee tool should inherit existing workforce identity and permissions.

Accessibility belongs in this layer. Captions, a text-input alternative, microphone controls, timeouts and a clear route to a human are not cosmetic additions. They affect whether the service is usable and governable.

2. The real-time media layer

Audio and rendered video usually need low-latency transport, often through WebRTC or a comparable real-time protocol. The bank must decide where sessions terminate, which network routes are allowed and whether audio or video is ever recorded.

Firewalls, proxies, browser policies and restricted networks can introduce more delay or break behaviour that worked in a lab. Test with the real branch network, approved browsers and production security controls early. Yepic’s work with Abu Dhabi Aviation and Oracle included development and production environments, real-time streaming, microphone and caption behaviour, WebRTC and network testing, browser remediation and cybersecurity support. That experience is relevant because enterprise integration quality is usually determined at the boundaries between systems, not in a perfect demo environment.

3. The conversation and policy layer

This layer coordinates speech recognition, retrieval, the language model, business rules and response generation. In a banking design, it should do more than produce plausible language. It should control what sources can be used, which actions are permitted and what happens when confidence is low.

Separate informational answers from transactions. An avatar explaining mortgage documents is a different risk class from one changing an address or initiating a payment. Higher-risk actions should pass through the bank’s existing authentication, authorisation and confirmation controls rather than relying on the conversational model.

4. The rendering layer on private GPUs

The rendering service turns generated speech and behavioural signals into a responsive digital person. Running this layer on customer-controlled GPUs can keep media generation inside the chosen environment and reduce reliance on a shared public service.

It also creates operational responsibilities. GPU memory, model loading time, concurrent sessions, resolution, frame rate, failover and headroom all need capacity planning. A peak branch hour is not the moment to discover that a visually impressive single-user test cannot sustain production concurrency.

Yepic has spent years developing proprietary talking-photo, AI-video and real-time avatar technology, including optimisation for commercial GPUs and sub-second multimodal interaction in appropriate deployments. Those capabilities do not remove the need to benchmark the complete customer architecture. Speech, retrieval, language-model inference, network paths and rendering all contribute to perceived response time.

5. Enterprise data and integration

Put an API gateway or equivalent policy layer between the conversational service and bank systems. The avatar should receive only the data needed for the current task, under the permissions of the authenticated user. Avoid giving the conversational stack broad database access merely because it is inside the network.

Typical integrations may include an approved knowledge base, product catalogue, appointment booking, customer identity, case management or contact-centre handoff. Each connector needs an owner, schema, access policy, timeout, error behaviour and audit trail.

6. Operations and observability

A bank needs to know whether the service is healthy without unnecessarily collecting customer conversations. Separate operational telemetry from interaction content. Teams may need latency percentiles, error types, component health, GPU utilisation and session counts; they may not need raw audio or full transcripts.

The operating model should also cover model and software updates, vulnerability remediation, configuration changes, rollback, key rotation, incident response and disaster recovery. The UK National Cyber Security Centre’s guidance treats secure deployment and secure operation as distinct, continuing responsibilities, including infrastructure protection, incident management, monitoring and update management.

What should remain inside the bank’s environment?

There is no single correct boundary, but a regulated deployment should make an explicit decision for every data class and component.

• Raw microphone and camera streams: keep local when policy or the use case makes them sensitive; avoid recording by default unless there is a justified purpose.
• Transcripts and generated responses: define retention, redaction, access and deletion rules rather than treating them as ordinary application logs.
• Customer identifiers and session state: keep within the bank’s identity and session controls, with short-lived, purpose-limited tokens.
• Knowledge and retrieved context: use approved repositories and enforce document-level permissions before information reaches the model.
• Model inputs and outputs: document whether any external model endpoint receives them and what the provider may retain.
• Avatar rendering: private-GPU inference can keep generated voice and imagery within the controlled environment.
• Operational logs: collect enough to diagnose performance and security events while minimising conversational content.

Some architectures may still use an external service for model distribution, licence validation, support telemetry, speech or language inference. That can be acceptable when it is intentional, contractually understood and compatible with policy. It is not accurate to call a system fully isolated if a necessary runtime component still makes external calls.

Being on-premise does not create zero trust

A private network is a deployment location, not a security decision. NIST’s Zero Trust Architecture guidance says trust should not be granted solely because a user, device or service is inside a particular network. Apply the same principle between the avatar orchestrator, models, vector store, API gateway and business systems.

Use workload identities, mutual authentication where appropriate, least-privilege service accounts, explicit authorisation and short-lived credentials. Segment components according to risk, but do not assume segmentation alone proves an interaction is authorised.

Build an end-to-end latency budget

Users experience one delay, even though architects see many components. Break the response path into measurable stages:

1. audio capture and network transport;
2. speech detection and recognition;
3. retrieval and policy checks;
4. language-model generation;
5. text-to-speech;
6. avatar rendering and encoding; and
7. return transport, buffering and display.

Measure time to first response as well as full completion. Streaming can make an interaction feel faster, but only if downstream components can begin work before the entire upstream output is complete. Co-locating dependent services may reduce network delay, while larger models or higher-fidelity rendering may consume more compute. The right trade-off depends on the job the avatar is doing.

Ten questions for the architecture review

1. Where is the trust boundary? Mark every runtime component, data store and outbound connection.
2. Which identity acts? Distinguish the end user, the channel, each workload and support personnel.
3. What content is retained? Set separate policies for audio, video, transcripts, prompts, responses and telemetry.
4. Who controls keys and secrets? Use the bank’s established custody, rotation and privileged-access processes.
5. What is the concurrency target? Model ordinary and peak sessions, not only one successful demonstration.
6. How does the service fail? Design timeouts, safe answers, degraded modes and human handoff.
7. How are updates delivered? Define approval, testing, signing, promotion and rollback for software, models and avatar assets.
8. Which components require internet access? Validate dependencies in the production network before accepting an isolation claim.
9. What can auditors reconstruct? Record decisions and authorised actions without collecting unnecessary sensitive content.
10. Who operates each layer? Assign monitoring, patching, capacity, incident and vendor-support responsibilities.

When private cloud may be the better answer

On-premise can offer direct control over infrastructure, data paths and change windows. It may suit a bank with strict residency requirements, existing GPU capacity and teams able to operate the stack. It can also bring longer deployment cycles, hardware commitments and responsibility for availability and upgrades.

A private cloud can provide strong isolation and dedicated controls with more managed infrastructure. Public cloud can be appropriate for lower-risk uses, rapid pilots or variable demand. Hybrid designs are common: sensitive retrieval and policy stay private while a carefully assessed external service handles a less sensitive component.

The decision should follow data classification, risk, latency, scale and operating capability. For a broader decision framework, read on-premise versus private cloud versus public cloud AI avatars. Procurement teams can also use Yepic’s enterprise AI avatar buyer’s guide to test vendor claims.

What Yepic can credibly bring to a bank deployment

Yepic builds proprietary AI-video and real-time avatar technology rather than simply placing a user interface over another avatar provider. The platform has been used in demanding enterprise and government settings involving multilingual communication, real-time interaction, APIs, browser delivery and large-scale video generation.

The Abu Dhabi Aviation and Oracle enterprise avatar integration is useful evidence of the engineering around a production-facing avatar: separate development and production environments, API and iframe integration, live streaming, caption and microphone behaviour, network testing, browser remediation, cybersecurity work and ongoing maintenance.

That case study is not presented as a completed on-premise banking deployment. It demonstrates the integration discipline that a regulated deployment requires. Yepic’s private, on-premise and sovereign options are bespoke implementations, scoped against the customer’s hardware, network, model choices, security requirements and operating model. Fully air-gapped operation should never be assumed without validating every dependency.

Start with the boundary, not the avatar

A bank should begin its AI avatar programme with one bounded workflow, a classified data-flow diagram and measurable service targets. Decide what the avatar may know, what it may do, where each component runs and how a human takes over. Then test that design on the real network with realistic concurrency and failure conditions.

The face is the most visible part of the experience. The architecture behind it determines whether the service can earn trust.