8. AI and LLM Workload Patterns
Modules 06 and 07 established how to build a pseudonymized data platform — zones that separate data by privacy level, pipelines that move records between zones, dynamic data masking (DDM) that gives different users different views, and key management that keeps the cryptographic infrastructure secure. For structured data, this works well: columns are classified, transforms are applied per field, and access control is declarative.
AI and LLM workloads break this model. They process unstructured text — support tickets, contracts, chat transcripts — where PII is embedded in free-form content, not neatly in columns. An LLM summarizing a customer complaint doesn't read a customer_name column; it reads "János Kovács called from +36201234567 about his bill." Sensitive Data Protection (SDP) can inspect and de-identify free text — it's the same engine — but pipelines are often configured only for structured-field transforms, leaving free-text columns untouched. DDM doesn't mask words inside a free-text field. And even when the data is properly pseudonymized, the anonymization may degrade the LLM's output quality to the point of uselessness — because the linguistic structure the model needs has been destroyed.
This module covers three concerns that the structured-data architecture from Modules 06–07 doesn't address: how anonymization affects different AI workload types, what strategies preserve both privacy and model quality, and how to architect the GCP platform — from VPC-SC perimeters through Model Armor to enterprise AI products — so that LLMs can process sensitive data without leaking it. For the underlying detection and transformation tools (SDP, Presidio), see Module 05.
How anonymization affects AI workloads
Before choosing a mitigation strategy, we need to understand what breaks. Anonymization affects traditional ML and LLM workloads differently — and the distinction matters because the mitigation strategies diverge.
Traditional ML/NLP (fine-tuned classifiers, NER models, custom entity extractors) operates on training data and inference inputs. Anonymization affects both:
| Task type | Impact on training | Impact on inference | Practical approach |
|---|---|---|---|
| Classification (sentiment, topic, intent) | Low — models learn patterns, not entity surface forms | Low — replacing "John Smith" with [PERSON_NAME] barely affects classification | Train on anonymized data. Anonymize inference inputs. Minimal quality loss. |
| NER / entity extraction | High — the model needs real entity examples to learn recognition patterns | Counterproductive — anonymizing inputs removes the entities the model is supposed to find | Train on real data (with access controls); anonymize the output of the model, not the input. |
| Translation | Low to moderate — grammar is preserved even with placeholders | Low to moderate — gendered languages may lose agreement cues when names are replaced | Train on real data; anonymize inputs at inference time for non-sensitive deployments. |
For traditional ML, the standard approach is: pseudonymize the training data (so models can be trained without exposing raw PII to data scientists), but accept that some tasks (NER, entity extraction) require access to realistic entity forms during training. Use the pseudonymized zone with DDM (Module 06) — data scientists see hashed subscriber IDs but full text content.
LLM workloads (prompting Gemini, RAG, agentic workflows, conversational analytics) operate on prompts constructed from data and retrieval corpora. Anonymization affects different parts of the LLM interaction:
| Concern | Impact | Why |
|---|---|---|
| RAG retrieval quality | High | Replacing entities with placeholders disrupts embedding quality. A query about "customer complaints in Budapest" won't match documents where Budapest has been replaced with [LOCATION]. Semantic similarity degrades. |
| Prompt grounding | Moderate to high | When structured data is injected into a prompt (e.g., customer record for a support agent), placeholder tokens (a3f8b2...) give the LLM nothing to reason about. "Customer a3f8b2 in district [REDACTED]" is less useful than "Customer in North district." |
| Agent tool calls | Low to moderate | An agent querying BigQuery over pseudonymized data works fine — it operates on column values, not semantic meaning. HMAC-hashed subscriber IDs are joinable and queryable. |
| Conversational analytics | Low | Natural language queries over structured data are minimally affected — the LLM generates SQL, and DDM masks the results. The LLM reasons about column names and aggregation patterns, not individual PII values. |
| Summarization | Low to moderate | Summaries can reference placeholders. Quality degrades only if anonymization removes context structurally relevant to the summary. |
| Text generation for customers | High | Generated text with placeholder tokens is unacceptable for customer-facing output. Context-preserving pseudonymization (see below) or lawful processing of personal data is required. |
| Hallucination risk | Elevated | LLMs encountering unfamiliar tokens ([PERSON_NAME], hash values) are more likely to hallucinate surrounding details or treat placeholders as instructions. |
The key insight: for LLM workloads, the critical question is not "which NLP task" but "where does PII appear in the LLM interaction?" — in the retrieval corpus, in the prompt, in the response, or in tool call results. Each position has different mitigation strategies, covered in the architecture patterns below.
Free-text fields — where pipeline pseudonymization falls short
The impact table above assumes the data reaching the LLM has been properly pseudonymized. In practice, the most common failure is simpler: the pipeline pseudonymized the structured columns but missed the PII embedded in free-text fields.
SDP's content.deidentify can handle both structured fields and free text — it's the same API. But pipelines are typically configured with per-column transforms (MSISDN → HMAC, customer name → redacted, age → bucketed), and free-text columns are often passed through without an SDP text inspection pass. The result: many datasets contain free-text fields — support notes, complaint descriptions, chat transcripts, agent summaries, contract clauses — where the same PII appears embedded in unstructured text. A pipeline that pseudonymizes the msisdn column but passes the support_notes column through untouched will produce records like:
| subscriber_hash | support_notes |
|---|---|
a8f3b2... | "Customer János Kovács called from +36201234567 regarding billing dispute on account HU-98765..." |
The structured identifier is pseudonymized. The free text contains the customer's name, phone number, and account number in plain text. When this record enters an LLM prompt — for summarization, classification, or RAG retrieval — the PII goes with it.
This is a common blind spot, and free text isn't the only hidden channel — documents processed by RAG pipelines or indexed by Gemini Enterprise may contain PII in places that aren't visible on screen: PDF layers, invisible text, embedded metadata (author names, tracked changes, comments). A document that looks clean to a human reviewer may still leak PII through its metadata when an LLM ingests it.
Adding coverage:
- Inbound (prompt): Free-text fields injected into prompts must be scanned with SDP text inspection or Presidio NER before reaching the LLM. This is what the detect-then-process pipeline (Pattern 3) and Model Armor (Pattern 4) address — but only if the free-text fields pass through them. If the application constructs prompts directly from BigQuery without a scanning step, the PII reaches the LLM unfiltered.
- Outbound (response): The LLM may reproduce PII from its context in the generated response — summarizing a support ticket and including the customer's name verbatim. Model Armor screens responses as well as prompts; for custom pipelines, the post-processing stage must scan LLM output with the same SDP templates used for input.
Treat free-text columns as a separate PII detection pass in the pipeline. Apply SDP text inspection (or Presidio for NLP-heavy detection) to every free-text column, with transforms appropriate for the downstream use case — redaction for fields the LLM doesn't need, context-preserving pseudonymization for fields it does. This runs alongside the structured-field transforms, not instead of them.
A useful detection technique for validating that your pipeline actually catches leakage: canary tokens — inject known, trackable fake PII values (a synthetic phone number, a dummy name) into prompts and monitor whether they appear in LLM responses. If the canary surfaces in the output, your outbound scanning has a gap. This is a lightweight test you can run in staging or as a periodic production health check.
Preserving model quality — selective masking and context-preserving pseudonymization
The previous sections showed what breaks and where the gaps are. The natural response — "anonymize everything before it reaches the LLM" — works for structured data but can make LLM output useless when applied to free text. Two strategies preserve both privacy and utility without the all-or-nothing trade-off:
Selective masking
Selective masking masks only what is irrelevant to the LLM task, leaving task-relevant fields intact. This requires understanding what the LLM needs:
| Use case | Mask | Preserve |
|---|---|---|
| Sentiment analysis on support tickets | Names, account numbers, phone numbers | Complaint text, product references, location (if relevant to regional analysis) |
| RAG over incident reports | Personnel names, internal IDs | Technical details, timestamps, system names, locations |
| Semantic search over contracts | Signatory names, account numbers | Contract terms, obligations, entity roles (buyer/seller) |
| Agent summarizing customer history | Account numbers, addresses | Service interactions, complaint themes, product names |
Selective masking is a risk-based decision, not a technical default. It requires agreement between the data engineer, the privacy team, and the AI team on which fields carry privacy risk vs. task value. Document the rationale — which fields are masked and why — for audit purposes.
Context-preserving pseudonymization
Context-preserving pseudonymization replaces identifiers with realistic substitutes rather than tokens or redaction markers. Instead of replacing "John Smith" with [PERSON_NAME], replace it with "David Wilson." Instead of [LOCATION], use "Springfield."
This preserves:
- Linguistic structure — the sentence reads naturally, preserving grammar, coreference, and context
- Embedding quality — a realistic name occupies a similar position in embedding space as the original, maintaining RAG retrieval accuracy
- Model behavior — LLMs perform better on natural language than on text peppered with placeholder tokens, reducing hallucination risk
The trade-off: context-preserving pseudonymization requires a consistent substitution strategy (the same "John Smith" maps to the same "David Wilson" throughout a document set, for coreference consistency), and the substitute must not collide with a real individual in the dataset.
Implementation: Presidio supports custom operators for context-preserving replacement. For GCP-native pipelines, combine SDP detection (to find the entities) with a custom Beam transform (to replace with realistic substitutes from a lookup table or a generative model).
Architecture patterns for LLM processing in data zones
With the problem understood and the strategies defined, the remaining question is how to implement them on GCP. LLM workloads typically operate in the pseudonymized zone, needing individual-level records with semantic content preserved, not structurally anonymized aggregates.
The six patterns below form a defense-in-depth stack, not a pick-one menu. They layer from infrastructure up:
- VPC-SC perimeter — network-level containment (prevents data exfiltration)
- Prompt-level data minimization — application-level reduction (send less data to the LLM)
- Detect-then-process pipeline — custom pipeline-level PII handling (full control, context-preserving)
- Model Armor — managed pipeline-level PII handling (same SDP templates, less code)
- Gemini Enterprise — enterprise search product with built-in DLP (safety net, not primary control)
- BigQuery Conversational Analytics — natural language queries with DDM (inherits existing access control)
A production deployment typically combines patterns 1 + 2 + (3 or 4) + whichever products (5, 6) the engagement uses.
Pattern 1: VPC-SC perimeter — network containment
VPC Service Controls perimeter
├── BigQuery (pseudonymized zone)
│ └── Source data with pseudonymized identifiers
├── Agent Platform (formerly Vertex AI)
│ └── LLM endpoints (Gemini, fine-tuned models)
│ └── No-logging, no-training guarantees
│ └── Model Armor with SDP templates (see Pattern 4)
├── Cloud Run / GKE
│ └── Application logic (prompt construction, post-processing)
└── Cloud Storage
└── Intermediate results (within perimeter)
The VPC-SC perimeter ensures that pseudonymized data cannot be exfiltrated to projects outside the perimeter — even by a principal with IAM permissions. This is the primary defense against accidental data leakage through LLM APIs.
Key configuration for Agent Platform within a perimeter:
- No data training — Google does not use customer data (prompts, responses) to train models without explicit permission. This is contractual, not just a configuration flag.
- Data residency — use regional endpoints to ensure data stays within the required jurisdiction.
- Data retention — by default, Agent Platform may cache prompts and responses in-memory for performance (project-isolated, short TTL) and retain limited data for abuse monitoring. Retention specifics vary by feature and configuration; consult the current data retention documentation for the applicable terms. For sensitive workloads, configure the strictest available retention settings and verify compatibility with all enabled features.
- Confidential Computing (when available) — Confidential VMs or Confidential GKE nodes process data in encrypted memory, providing hardware-level isolation.
Pattern 2: Prompt-level data minimization — send less
VPC-SC prevents data from leaving the perimeter, but it doesn't control what goes into the LLM prompt within the perimeter. Prompt-level data minimization reduces exposure at the application layer — construct prompts that contain only the minimum data needed for the task:
Full record: {subscriber_id_hash: "a3f8b2", age_band: "30-39",
district: "North", complaint_text: "My bill is wrong...",
account_number: "HU12345678", name: "János Kovács"}
Minimized prompt: "Classify the sentiment of this customer complaint:
'My bill is wrong...'"
The prompt includes only the complaint text — the LLM never sees the subscriber hash, age band, district, account number, or name. The classification result is joined back to the record in the application layer.
Pattern 3: Detect-then-process pipeline — custom PII handling
For tasks that require semantic content (RAG, summarization), run the anonymization in two stages:
Source text
→ Stage 1: PII detection (SDP + Presidio NER)
→ Produce entity map: [(span, entity_type, original_value)]
→ Stage 2: Context-preserving replacement
→ Replace entities with realistic substitutes
→ Maintain substitution map for post-processing reversal
→ Stage 3: LLM processing
→ LLM operates on pseudonymized text with natural language structure preserved
→ Stage 4: Post-processing
→ Optionally reverse substitutions in LLM output for authorized consumers
This pipeline preserves semantic quality for the LLM while ensuring that the text entering the LLM contains no real PII. The substitution map contains the original PII values and reverses every pseudonym, so it must not be stored in the pseudonymized zone alongside the data it protects — that would give the pseudonymized zone a direct re-identification path. Store it in a separate key-management dataset (or project) with narrowly scoped re-identification access, following the same access patterns as the DEK table described in Module 07.
Pattern 4: Model Armor — managed PII handling
Model Armor provides a managed alternative to building a custom detect-then-process pipeline. It intercepts prompts before they reach the LLM and responses before they reach the application, screening both for PII using the same SDP engine and templates used in data pipelines.
Application
→ Prompt → Model Armor (SDP inspect + de-identify)
→ Sanitized prompt → Gemini model
→ Response → Model Armor (SDP inspect + de-identify)
→ Sanitized response → Application
Model Armor operates in two modes:
| Mode | Configuration | Behavior |
|---|---|---|
| Basic | Predefined InfoTypes (mostly US-focused) | Detects common PII patterns out of the box. Sufficient for quick deployment. |
| Advanced | Your existing SDP inspect template + optional de-identify template | Uses the same custom InfoTypes (MSISDN, IMSI patterns) and transforms (FPE, HMAC, redaction) as your data pipelines. Single source of truth for PII detection across the entire architecture. |
In advanced mode with both inspect and de-identify templates configured, Model Armor returns a sanitized version of the prompt/response with transforms applied — the application can send the sanitized prompt to the LLM without any custom PII-handling code. If only an inspect template is configured, Model Armor reports findings and the application decides whether to block, log, or proceed.
Model Armor vs. custom detect-then-process pipeline:
| Dimension | Model Armor | Custom pipeline (Pattern 3) |
|---|---|---|
| Setup effort | Configuration only — point at existing SDP templates | Custom Beam transforms, substitution map management |
| Context preservation | Limited — supports SDP de-identify transforms (redaction, hashing, masking) but not realistic-substitute replacement | Full — can replace entities with realistic substitutes for embedding quality |
| Integration | Native on Agent Platform; also available as a standalone API for non-GCP LLM providers | Requires custom pipeline code |
| Runtime overhead | Managed — Google handles scaling and latency | Self-managed — scales with pipeline infrastructure |
| Best for | Classification, summarization, and other tasks tolerant of standard masking | RAG, semantic search, and tasks requiring natural-language-preserving pseudonymization |
For most LLM workloads, start with Model Armor. When integrated natively via Agent Platform's gateway, it is configuration-only — no custom PII-handling code. For custom applications outside the gateway, the application must route both prompts and responses through Model Armor's sanitize API, which requires integration code but no custom PII detection logic. Escalate to a custom detect-then-process pipeline only when the LLM task requires context-preserving pseudonymization that Model Armor's SDP transforms cannot provide.
Pattern 5: Gemini Enterprise — enterprise search with DLP safety net
Patterns 1–4 cover custom AI pipelines where engineers control the architecture. The remaining two patterns cover GCP enterprise AI products where the privacy controls are built into the product itself.
Gemini Enterprise (formerly Agentspace) is Google's enterprise AI assistant — a chat-based interface that searches and reasons over organizational data via connectors (Google Drive, BigQuery, Jira, Salesforce, Confluence, and others). When the search corpus contains data with PII, Gemini Enterprise provides several privacy controls, but it is not an anonymization tool — it is a last-line-of-defense that prevents PII from surfacing to unauthorized users.
What Gemini Enterprise handles:
- Document-level access control — enforces source permissions. If a user cannot access a Drive file or Jira ticket in the source system, Gemini will not retrieve it.
- DLP/SDP filtering on responses — blocks responses that trigger SDP filters. If the model generates output containing detected PII, the response is blocked before reaching the user.
- IRM enforcement — files with information rights management controls (prevent download, copy, print) are excluded from retrieval entirely.
- Client-side encryption (CSE) — CSE-encrypted content is indecipherable to Gemini. The data is never indexed or searchable.
- Custom block lists — organization-specific patterns beyond built-in SDP InfoTypes.
- No training on customer data — prompts and responses are not used to train models.
What Gemini Enterprise does NOT handle:
- Pre-ingestion anonymization. Gemini Enterprise indexes data from connectors as-is. It does not transform, pseudonymize, or anonymize data before indexing. If the source data contains raw PII, the search index contains raw PII.
- Context-preserving pseudonymization. No realistic-substitute replacement for preserving search quality.
- Structural anonymization. No k-anonymity, DP, or dataset-level privacy operations.
The recommended pattern: anonymize or pseudonymize data in the pipeline (Module 06) before it enters the Gemini Enterprise search corpus. Gemini Enterprise's DLP/SDP filters then act as a safety net — catching anything the pipeline missed or that leaks through retrieval-augmented generation.
Source data
→ Pipeline pseudonymization (SDP transforms)
→ Pseudonymized data store (BigQuery, Drive, etc.)
→ Gemini Enterprise connector indexes pseudonymized data
→ User query → Gemini retrieves from pseudonymized corpus
→ Response → DLP/SDP filter (safety net) → User
This layered approach — pipeline anonymization as the primary control, Gemini Enterprise DLP as the secondary — ensures that PII exposure requires failures in both layers simultaneously.
Pattern 6: BigQuery Conversational Analytics — DDM does the work
BigQuery Conversational Analytics lets users ask natural language questions about their BigQuery data ("what were the top 10 districts by call volume last month?"). A Gemini-powered data agent generates SQL, executes it, and returns answers with charts and reasoning — supporting multi-turn follow-up questions, forecasting, and anomaly detection.
The privacy-relevant property: generated queries execute under the querying user's identity. This means the entire DDM and column-level access control stack from Module 06's query-time masking section applies automatically:
- A user with
maskedReaderon policy-tagged columns gets answers based on masked values — the LLM sees hashed subscriber IDs, not raw MSISDNs - A user with no access to a tagged column gets a query error if the conversation touches it
- Queries are read-only — Conversational Analytics cannot run DML, eliminating accidental data modification
This is the use case where the pseudonymized zone + DDM design (Module 06) pays off directly. Business analysts ask natural language questions about pseudonymized data and get useful answers without ever seeing raw identifiers — no additional anonymization layer between the user and the data, no Model Armor configuration, no custom pipeline. The policy tags and masking rules that govern regular SQL queries govern the AI-generated queries identically.
The limitation: Conversational Analytics operates on whatever data the user can see. If the user has fineGrainedReader (raw access), the LLM sees raw data too. The privacy guarantee is only as strong as the IAM configuration — which is true for all DDM-based patterns, but worth emphasizing because natural language interfaces lower the barrier to exploratory querying. An analyst who wouldn't think to write SELECT msisdn FROM cdrs might naturally ask "show me the phone numbers of customers who churned" — and if their IAM role grants raw access, they'll get them.
For custom AI pipelines on Agent Platform, start with Model Armor (Pattern 4) — it reuses your existing SDP templates with no custom code, and handles most classification and summarization workloads. Escalate to a custom detect-then-process pipeline (Pattern 3) only for RAG and semantic search where context-preserving pseudonymization is needed. For enterprise search (Gemini Enterprise), anonymize the data before it enters the search corpus — Gemini Enterprise's DLP controls are a safety net, not the primary anonymization layer. For BigQuery natural language analytics (Pattern 6), the existing DDM configuration does the work — but audit IAM roles carefully, because natural language interfaces make it easier for users to request data they technically have access to but wouldn't normally query. In all cases, VPC-SC perimeters (Pattern 1) and prompt-level data minimization (Pattern 2) are defense-in-depth measures that apply regardless of which LLM integration you choose.
Next: Module 09 covers governance and compliance — the policies, roles, legal frameworks, and organizational structures that make anonymization decisions consistent and defensible. Module 10 then covers the technical machinery that enforces those decisions: automated classification, pipeline monitoring, lineage, and incident response.
Created by Claude Opus with ❤️ for the Aliz Data Engineering team.