AI Data Classification: Decide What an Avatar May Process

AI data classification should decide what every component in a real-time avatar may process, where it may run, what it may retain and which actions it may support. A label on a source document is useful, but it is not enough. During one conversation, an avatar can combine microphone audio, identity, retrieved knowledge, generated text, synthetic voice, operational telemetry and a request to change a business record. The handling rule must survive that entire journey.

For a bank, government department or regulated enterprise, the practical deliverable is a data-class contract enforced at the boundaries between capture, speech recognition, retrieval, language inference, voice and image generation, business actions, logging and support. If a classification does not change a routing or control decision, it is merely descriptive metadata.

Classify the live interaction, not just the documents

Conventional information-classification programmes tend to start with files, email, databases and records. A real-time avatar introduces transient and derived information that may never become a document:

• partial speech transcripts held for milliseconds or seconds;
• account identity, entitlements and session context;
• passages retrieved from differently classified sources;
• inferences about intent, sentiment or urgency;
• model responses, captions, synthetic speech and rendered frames;
• proposed or executed actions in a system of record; and
• traces, error payloads, recordings and support bundles.

The first architecture step is therefore a component-level map of the avatar’s data flows. Classification adds a decision to every row: which handling policy applies, which components are permitted and what must happen if the required route is unavailable.

The UK Government Security Classifications Policy, updated on 5 August 2024, illustrates the underlying principle. Its OFFICIAL, SECRET and TOP SECRET tiers connect potential impact and threat profile to baseline behaviours and controls wherever information is collected, stored, processed or shared. It is an HMG administrative scheme, not a universal enterprise vocabulary, but the important lesson is portable: classification should drive protection across the lifecycle, not simply add a marking. See the current Government Security Classifications Policy.

Use four dimensions rather than one overloaded label

A single ladder such as Public, Internal, Confidential and Restricted can be a helpful starting point. It becomes unreliable when asked to express every risk. Separate four dimensions so architecture teams can make precise decisions.

1. Content sensitivity

What harm could follow from unauthorised disclosure or misuse? Classify raw audio, transcripts, knowledge, generated media, logs and support artefacts separately. Public source material does not make an account-linked conversation public.

2. Identity and context

Is the interaction anonymous, pseudonymous, authenticated or tied to a regulated relationship? The same question about an interest rate has different handling implications when asked on a public kiosk and when combined with a named customer’s balances. Health information, biometric data and other legally protected categories require specific legal analysis; a security label does not supply a lawful basis.

3. Consequence and authority

What can the interaction cause? Distinguish an avatar that explains a policy from one that prepares an application, changes an address, books travel or moves money. The action ceiling may need to be more restrictive than the conversational content. A model must not infer its own authority from the user’s language or apparent emotion.

4. Residency and handling constraints

Which jurisdictions, deployment zones, processors and human roles are approved? Record whether a data class may use a public service, an approved private-cloud endpoint or only a customer-controlled environment. Add retention, export, support and recovery constraints. “Confidential” alone does not answer any of these questions.

NIST’s FIPS 199 separates potential impact on confidentiality, integrity and availability, while SP 800-60 Volume 1 maps information types to security categories. These are US federal references rather than ready-made rules for every organisation. They are useful reminders that disclosure is not the only consequence: an incorrect or unavailable citizen-service answer can matter even when its content is not secret.

Build a data-class contract that systems can enforce

Create one controlled record for each meaningful class and workflow. At minimum, include:

• class identifier, business owner and policy version;
• example data types and prohibited fields;
• approved purposes and user populations;
• permitted speech, language, retrieval, voice and rendering components;
• allowed deployment zones, regions and subprocessors;
• identity requirements and action ceiling;
• retention, logging, backup and support rules;
• permitted egress destinations and payload fields;
• safe fallback when an approved component is unavailable;
• rules for raising or lowering classification; and
• named approver, review date and evidence tests.

Existing sensitivity labels can seed this contract. Microsoft, for example, documents labels that remain in file and email metadata and can trigger encryption or permissions in supported services. Its guidance also makes an important distinction: a label can exist without protection settings. An avatar therefore needs an adapter that converts an enterprise label into pipeline policy; the presence of a Microsoft Purview sensitivity label does not automatically govern every speech, model or rendering service.

Enforce classification at seven gates

Gate 1: capture and session creation

Assign a default class from the channel, workflow and authentication state before opening the microphone. A public information kiosk might begin at a low content class but still require strict rules for accidental personal disclosures. An authenticated banking journey should begin with account-linked handling controls even before sensitive details are spoken.

Gate 2: speech and perception

Route audio only to an approved speech-recognition component. Preserve the class on partial transcripts, language detection and translation. Multilingual processing must not quietly introduce a remote service that the English path does not use.

Gate 3: retrieval

Combine source classification with the user’s current permissions. The retriever should return only authorised passages and attach source identity, revision and class to the result. The effective handling requirement for the next stage should reflect both conversation context and retrieved material.

Gate 4: language inference

Select an approved model endpoint and reduce the prompt to what is necessary. Do not assume that removing a name makes the remaining context harmless. Transactions, locations and rare combinations can re-identify a person. Unknown classification should produce a bounded refusal, clarification or human handover rather than an opportunistic external call.

Gate 5: presentation

Apply the same policy to text-to-speech, avatar rendering, captions and screen content. A response suitable for a private headset may be inappropriate on a five-metre display or an audible public kiosk. Classification should influence output channel, volume, masking and whether the answer can be displayed at all.

Gate 6: tools and business actions

Keep action authorisation outside the language model. A trusted gateway should check user identity, workflow, classification and action ceiling; constrain fields; require approval where necessary; and return an evidence receipt. A valid answer does not imply permission to execute the action it describes.

Gate 7: evidence, operations and support

Propagate the class into logs, traces, recordings, exports, incident evidence and support bundles. Prefer structured decision records over full conversational content. The privacy-minimised AI audit-trail design shows how to retain identities, versions, policy outcomes and action receipts without creating a second copy of every conversation.

The NCSC advises organisations to know what data they hold, where it is stored and what is most sensitive, then protect copies, interfaces, access and monitoring according to risk. Its data-security guidance also warns that replicated and cached copies need sufficient protection. That matters for avatar systems because a short-lived transcript can persist unexpectedly in a retry queue, vector index, trace or restored backup.

Let the classification change during the conversation

A session is not born with one permanent label. It can move from a public question to an authenticated account journey, retrieve a more sensitive record, propose a consequential action and then return to general guidance.

Define upward transitions that take effect before the sensitive turn is processed. They may require stronger authentication, a local component, a private output channel or a new session boundary. If the required control is unavailable, degrade safely.

Downward transitions deserve more caution. Redaction, aggregation or completion of a transaction may justify a lower class, but only through an explicit rule with evidence. Clearing the visible chat does not prove that model context, caches, logs and action tokens were cleared. Record who or what authorised the downgrade and which derived artefacts remain.

Choose deployment from the handling rule

Customer-hosted inference can give a bank or public body direct control over GPUs, networks, keys, logs and support access. It can keep selected speech, language, voice and rendering stages inside the customer environment when every chosen component supports that design. It also moves more responsibility for capacity, patching, recovery and operational assurance to the customer and implementation team.

Private cloud may combine dedicated resources, controlled networking and regional placement with managed infrastructure. Public cloud can be the sensible choice for public information, a time-limited pilot or volatile demand where contracts and technical controls satisfy the classification. Hybrid designs can route lower classes to managed components and keep higher classes private, but only when outbound data controls prevent silent fallback across the boundary.

On-premise is therefore an enforcement option, not a classification. It does not decide which data is acceptable, prevent over-broad administrator access or stop content-rich logs by itself.

Use project evidence carefully

Yepic’s Abu Dhabi Aviation and Oracle enterprise-avatar integration connected authorised users to operational information through a role- and clearance-aware layer. Delivery included separate development and production environments, API and iframe integration, real-time streaming, captions and microphone behaviour, WebRTC and corporate-network testing, browser remediation, cybersecurity support and ongoing maintenance.

That work demonstrates why identity, operational data, media behaviour and environment boundaries must be designed together. It is not presented as proof of the classification framework in this article or as a completed customer-hosted deployment. Yepic can scope cloud, private-cloud, sovereign and customer-hosted avatar systems, including deployments on customer GPUs, but the actual component placement and controls must be validated for each implementation.

Twelve tests for an architecture review

1. Every raw, derived, generated and operational data type has an owner and class.
2. An unclassified input fails safely instead of reaching a default public service.
3. Authentication changes the session policy before account-linked data is processed.
4. Retrieved passages carry source, revision, permission and classification metadata.
5. Translation and multilingual paths use the same approved handling boundary.
6. A higher-class turn cannot fall back to a lower-assurance component during failure.
7. Captions, speech and display rules match the privacy of the physical channel.
8. The model cannot increase its own action authority.
9. Logs, traces and support bundles contain only fields permitted for their class.
10. Downgrading requires an approved rule and leaves evidence.
11. Backup, disaster-recovery and support copies preserve the required controls.
12. A production test proves that policy labels change actual routes, retention and access.

Make the label change the system

Start with one bounded workflow and three representative conversations: public information, authenticated service and a high-consequence exception. Classify every input, derived artefact, output and action. Then complete the data-class contract and test each gate during normal operation, failure, support and recovery.

The decisive question for procurement is not “Does the platform support confidential data?” It is: “When this session becomes confidential, which components, people, locations and actions become unavailable—and what evidence proves that change happened?” A useful AI data-classification policy can answer that question before the avatar enters production.