Glossary
Terms used across this knowledge base, written for data engineers. Terms from all learning path modules (01β13) are included.
Aggregation. Collapsing individual records into group-level statistics (e.g., "1,247 calls in cell cluster X between 14:00β15:00" instead of individual CDRs). Stronger privacy than generalization β individual records no longer exist β but eliminates record-level analysis.
AMLA (Anti-Money Laundering Authority). The EU's first centralized AML/CFT supervisory authority, established in Frankfurt in 2025. Will directly supervise the highest-risk cross-border financial institutions starting 2028. Aims to harmonize AML enforcement across member states. Detailed in Module 11.
AMLD (Anti-Money Laundering Directive). EU directives governing customer due diligence, transaction monitoring, and suspicious activity reporting for financial institutions. AMLD5 (in force) requires 5-year retention of identifiable KYC records and transaction data (Art. 40); member states may extend to 10 years. The criminal-liability directive is Directive 2018/1673 (often called "6AMLD"). Directive 2024/1640 establishes the preventive AML/CFT framework covering national mechanisms, supervisors, and FIUs. Cross-institutional information-sharing partnerships between obliged entities are governed by AMLR Article 75. Being supplemented by the directly applicable AMLR. Detailed in Module 11.
AMLR (Anti-Money Laundering Regulation). EU Regulation 2024/1624 β a directly applicable "single rulebook" that will replace the patchwork of national AMLD implementations. Adopted in 2024, applying from July 2027. Establishes a uniform 5-year retention period for CDD and transaction records (Art. 77(3)). Until application, national AMLD transpositions continue to govern retention periods. Detailed in Module 11.
Amnesia. Web-based anonymization tool (OpenAIRE project) providing a graphical interface for k-anonymity on tabular data. Simpler than ARX, suited for workshops and pilot projects with non-technical users. Not designed for pipeline integration.
Anonymization. Transforming data so that no individual is identifiable by any means reasonably likely to be used β including by the data controller. Irreversible by design. Under GDPR, anonymized data exits regulation scope entirely. Contrast with pseudonymization.
Anonymization policy. An organizational document defining scope, classification scheme, approved techniques, zone mappings, retention rules, exception handling, and review cadence for anonymization across a data platform. Creates a single source of truth so technique selection and data handling are consistent across teams. A ready-to-adapt template is included in this knowledge base. Detailed in Module 09.
Apache Ranger. Open-source centralized access control framework providing tag-based, role-aware column masking and row filtering for Hadoop/Spark ecosystems (Hive, Spark SQL, Presto, Trino). The on-premises equivalent of BigQuery column-level security.
ARX. Open-source Java framework for tabular data anonymization. Supports k-anonymity, l-diversity, t-closeness, differential privacy. Computes re-identification risk under prosecutor, journalist, and marketer models. Detailed in Module 05.
Aspect (Knowledge Catalog). Structured metadata attached to a catalog entry β the successor to Data Catalog tags. Sensitive Data Protection publishes discovery profiles as aspects, containing sensitivity level, risk level, and detected InfoTypes per resource.
Authorized view. (BigQuery) A view with its own IAM permissions that users query instead of the underlying table. Used for row-level filtering when the logic is more complex than a row-level access policy β e.g., joining against a permissions table, applying business rules, or aggregating before exposure. Complements column-level masking (DDM) in the pseudonymized zone.
Auxiliary dataset. An external data source an attacker can cross-reference with anonymized data to re-identify individuals β voter rolls, social media, previous data breaches, commercial data broker databases.
Background knowledge. What an attacker already knows about individuals in a dataset β home location, workplace, habits. Reduces the number of quasi-identifiers needed for re-identification.
Bank client data. (FINMA) All information protected by Swiss banking secrecy β broader than standard PII, covering account balances, transaction details, and any information about a banking relationship. Defined in FINMA Circular 2023/1.
BIN. Bank Identification Number β the first 6β8 digits of a credit/debit card number, identifying the issuing bank. A quasi-identifier (narrows the issuer, not the individual).
Canary token. A synthetic, trackable fake PII value (dummy phone number, fake name) injected into LLM prompts to test whether outbound scanning catches PII leakage. If the canary appears in the LLM response, the output filtering has a gap. A lightweight validation technique for staging or periodic production health checks. Detailed in Module 08.
CCPA/CPRA. California Consumer Privacy Act / California Privacy Rights Act. Extends to household-level data and inferences drawn from personal information. Civil penalties.
CDPA (Cloud Data Processing Addendum). Google Cloud's contractual DPA (Article 28) governing the processing of customer data. Key commitments: customer data is not used to train Google's foundation models (paid Gemini API, Vertex AI), data remains in the customer's project during fine-tuning, and processing occurs only per CDPA terms. Required when processing personal data through GCP AI services. Detailed in Module 09.
CIS Benchmark for GCP. Security configuration guidelines published by the Center for Internet Security. Control 1.10 recommends 90-day rotation for Cloud KMS encryption keys β the default that GCP tooling encourages and the most commonly cited rotation period for pseudonymization KEKs. Detailed in Module 10.
Classify then enforce. A GCP governance pattern where data is profiled automatically (via SDP discovery), tagged with a sensitivity level, and access controls are applied based on the tag. New data is locked down by default until classified and reviewed by a data steward. Detailed in Module 10.
CDR. Call Detail Record β a log of a single call or data session: who called whom, when, from which cell tower, for how long. Rich in quasi-identifiers. 4 spatio-temporal points from CDRs identify 95% of subscribers.
Central DP. A deployment model where a trusted curator holds raw data and adds noise to query outputs. The standard model for enterprise analytics β the data warehouse holds unmodified data, and a DP layer sits between the data and any external consumer. Contrast with local DP.
Cloud EKM. Google Cloud External Key Manager β encryption keys held outside Google's infrastructure (on-prem or third-party HSM) while used transparently with GCP services. Recommended for Swiss banking engagements where pseudonymization keys must be provably inaccessible to the cloud provider. Detailed in Module 05.
Cloud Audit Logs. GCP's logging service for administrative and data access events. Admin Activity logs (always on, 400-day retention) capture configuration changes β SDP template modifications, KMS key operations, IAM policy changes. Data Access logs (must be enabled for most services; enabled by default for BigQuery) capture data read/write operations. Essential for anonymization audit trails. Detailed in Module 10.
Cloud KMS. Google Cloud Key Management Service β manages encryption keys for GCP services. Supports software-backed keys, HSM-backed keys, and external keys (via Cloud EKM). Detailed in Module 05.
Composition attack. Combining multiple independently anonymized releases of overlapping data to narrow equivalence classes and re-identify individuals. Each release may individually satisfy k-anonymity, but their intersection can reduce effective k dramatically. Differential privacy resists composition via its composition theorem β cumulative privacy loss is bounded mathematically. Detailed in Module 12.
Context-preserving pseudonymization. Replacing identifiers with realistic substitutes rather than tokens or redaction markers β "John Smith" β "David Wilson" instead of [PERSON_NAME]. Preserves linguistic structure, embedding quality, and LLM behavior. Requires consistent substitution (same input β same substitute throughout a document set) and collision avoidance (substitutes must not match real individuals in the dataset). Detailed in Module 08.
Critical data. (FINMA) Data requiring heightened protection based on the institution's size, complexity, and risk profile, assessed across confidentiality, integrity, and availability. Defined in FINMA Circular 2023/1.
Crypto-shredding. Destroying a cryptographic key to render pseudonymized data unattributable (cannot be linked to a real-world identity) and undecryptable (pseudonyms cannot be reversed) β an alternative to locating and deleting every record for a data subject. Note: with deterministic pseudonyms, records sharing the same pseudonym value remain groupable even after key destruction; what is lost is the mapping to the original identity, not the intra-subject linkability. Detailed in Module 07.
Cryptoperiod. The time span during which a specific cryptographic key is authorized for use. NIST SP 800-57 recommends a maximum of 2 years for symmetric encryption and HMAC keys. PCI DSS v4.0.1 (Requirement 3.7.4) requires key retirement at the end of an organization-defined cryptoperiod based on industry guidance and data sensitivity β it does not prescribe a specific interval. The CIS Benchmark for GCP recommends 90 days for Cloud KMS keys. GDPR is silent on specific periods β left to the controller's risk assessment. Detailed in Module 10.
DAMA DMBOK. The Data Management Body of Knowledge, published by the Data Management Association International. Codifies the standard data governance roles β data owner, data steward, data custodian β widely adopted across industries. Referenced in Module 09 for anonymization governance role definitions.
Data clean room. An environment where two or more parties can jointly analyze their combined datasets without either party accessing the other's raw data. BigQuery Data Clean Rooms enforce analysis rules on shared-in-place data with native differential privacy via the DIFFERENTIAL_PRIVACY SQL clause. Confidential Space provides TEE-based computation for higher-assurance scenarios. Used for cross-organization analytics in advertising, retail brand/retailer collaboration, and cross-institutional fraud detection. Detailed in Module 11.
Data custodian. The engineering role responsible for operating the anonymization pipeline, managing key infrastructure, and monitoring transform health. Defined in the DAMA DMBOK framework. Contrast with data owner (business accountability) and data steward (classification and retention). Detailed in Module 09.
Data owner. The business role that defines what anonymized data can be used for, approves access grants, and is accountable for business decisions made on the data. Defined in the DAMA DMBOK framework. Distinct from data custodian (engineering operations) and DPO (privacy advisory/monitoring). Detailed in Module 09.
Data steward. The role that validates data classification, manages retention and deletion schedules, and bridges business requirements with technical implementation. In anonymization governance, the steward reviews automated SDP classifications and confirms or adjusts sensitivity tier assignments. Defined in the DAMA DMBOK framework. Detailed in Module 09.
Data zones. An architecture pattern where the same source data is transformed to different privacy levels for different consumers. Typical zones: raw (production only), pseudonymized (internal analytics), de-identified (BI/reporting), anonymized or DP-guaranteed (external sharing). Detailed in Module 06.
DDM. See dynamic data masking.
De-identification. Replacing identifiers and coarsening quasi-identifiers (e.g., exact age β age range). Goes further than pseudonymization but may still carry residual re-identification risk. Whether it qualifies as anonymization under GDPR depends on context.
De-identification template. (Sensitive Data Protection) A reusable, Terraform-manageable configuration specifying which transforms to apply to which InfoTypes. A single template can apply different transforms per field β FPE for MSISDN, bucketing for age, redaction for names.
Delta (Ξ΄). (Differential privacy) A failure probability β a small chance that the Ξ΅ guarantee does not hold. Set much smaller than 1/n (dataset size), e.g., Ξ΄ = 10β»β· for a million records. In practice, set once and rarely revisited.
Dictionary attack. Precomputing the hash of every possible input value (e.g., all phone numbers in a national numbering plan) and matching against hashes in a dataset. Breaks plain hashing of finite-domain identifiers in minutes. Prevented by keyed hashing (HMAC) or salting. Also called a rainbow table attack when precomputed lookup tables are used.
Differencing attack. Subtracting one aggregate query from another to isolate an individual's contribution β e.g., "total salary for department X" minus "total salary for department X excluding person Y" reveals Y's exact salary. When only averages and group sizes are available, the same derivation applies: Y = n Γ avg(X) β (nβ1) Γ avg(X\Y). Minimum group-size thresholds raise the bar but do not prevent the attack when both groups exceed the threshold while differing by one person. The primary defense is differential privacy with shared budget accounting across queries; query auditing for overlapping query patterns provides an additional detection layer.
Differential privacy (DP). Adds calibrated mathematical noise to query outputs or model training so that the presence or absence of any single individual's data makes at most an Ξ΅ (epsilon) difference in the output. Lower Ξ΅ = stronger privacy, more noise. Ξ΅ β€ 1.0 is a commonly cited starting point, but the actual guarantee depends on additional parameters (Ξ΄, sensitivity, composition budget). Detailed in Module 04.
Discovery service. (Sensitive Data Protection) Continuous, automated profiling of data assets across an organization. Profiles BigQuery tables, Cloud SQL databases, Cloud Storage buckets, and more. Publishes results to Knowledge Catalog as aspects. The automated version of a one-off PII inspection.
Direct identifier. A field that uniquely identifies an individual on its own. Examples: name, email, national ID, MSISDN (telco), IBAN (fintech), customer account ID (e-commerce). Must be removed, replaced, or encrypted.
DORA (Digital Operational Resilience Act). EU regulation effective January 2025 applying to ~22,000 financial entities. Requires ICT risk management frameworks covering data integrity and confidentiality, incident reporting, and digital resilience testing. Anonymization of test data is one compliance lever. Detailed in Module 11.
DP-SGD. Differentially Private Stochastic Gradient Descent β training a model with formal DP guarantees so that no individual training record influences the output beyond a bounded amount. Used when generating synthetic data for external sharing. Reduces output quality.
DPIA. Data Protection Impact Assessment. Required under GDPR Article 35 for processing likely to result in high risk. Must include: systematic description of processing operations, necessity and proportionality assessment, risk assessment for data subjects, and mitigation measures. For anonymization systems, the anonymization technique itself is the primary mitigation. DPIAs should be reviewed after model updates, new data sources, use case expansion, or regulatory changes. Article 29 Working Party criteria provide a 9-factor checklist for when a DPIA is required. Detailed in Modules 02 and 09.
DPO (Data Protection Officer). The role mandated by GDPR Article 37 for certain organizations (public authorities, large-scale systematic monitoring, large-scale processing of special categories). GDPR Article 39 defines the DPO as advisory and monitoring β the DPO informs, advises, monitors compliance, and provides guidance on DPIAs, but does not own processing decisions. Decision accountability remains with the controller (data owner, CDO, CPO). Article 38 requires DPO independence, which conflicts with assigning decision-making accountability. Others may designate a DPO voluntarily; SMEs may use an external DPO-as-a-service provider. Detailed in Module 09.
DPSynth. Google's open-source Python library for differentially private synthetic data generation (google/dpsynth). Supports tabular, relational, nested, sequence, and text data. Uses marginal-based mechanisms (AIM, MST, SWIFT) rather than DP-SGD, providing formal DP guarantees on the synthetic output. Offers in-memory mode (up to ~100M rows) and an Apache Beam pipeline mode for distributed processing on Dataflow. Under active development; not an officially supported Google product. Compare with SDV (no native DP) and Gretel (commercial, DP optional).
DSAR (Data Subject Access Request). A request from an individual exercising their GDPR rights β most commonly right to erasure (Article 17). In anonymization architectures with per-subject DEKs, fulfilled via crypto-shredding. With shared HMAC keys, physical record deletion is required instead. Detailed in Module 10.
Dynamic data masking (DDM). Query-time masking that transforms values based on the caller's identity and role β the stored data is never modified. In BigQuery: policy tags + masking rules, with Fine-Grained Reader seeing raw data and Masked Reader seeing masked output. An access control mechanism, not an anonymization technique. Contrast with static data masking.
Earth Mover's Distance (EMD). A distribution distance metric used by t-closeness. Measures how much "work" is needed to transform one distribution into another. Respects the semantic distance between values β moving probability mass between nearby values costs less than between distant values. Also known as Wasserstein distance.
EBIOS. Expression des Besoins et Identification des Objectifs de SΓ©curitΓ© β a general-purpose cybersecurity risk management framework widely used in European assessments. Structures risk as Severity Γ Likelihood. Can be applied to re-identification risk analysis but is not specific to privacy or statistical disclosure β the privacy-specific metrics (prosecutor/journalist/marketer risk) come from tools like ARX and sdcMicro.
EDPB. European Data Protection Board. Issues guidelines interpreting GDPR. Key references: Guidelines 01/2025 on pseudonymization, Guidelines 02/2026 on anonymisation (three-criteria test: no record isolation, no linkage, no inference).
Envelope encryption. A key management pattern that separates the key touching data (DEK) from the key protecting it (KEK). The DEK is encrypted by a KEK in Cloud KMS; at runtime, the DEK is decrypted, used for transforms, then discarded from memory. The KEK never leaves the HSM.
ePrivacy Directive (2002/58/EC). Layers on top of GDPR for electronic communications. Traffic data must be erased or anonymized when no longer needed; location data requires explicit consent. No "legitimate interest" fallback.
Equivalence class. A group of records that share identical values on all quasi-identifier fields. In k-anonymity, every equivalence class must contain at least k records β each individual "hides in a crowd" of at least k.
FADP (nDSG). Swiss Federal Act on Data Protection (revised, in force since September 2023). Aligns with GDPR. Pseudonymized data is covered if re-identification is possible with reasonable effort. Operates alongside banking secrecy for financial institutions.
Federated learning. A machine learning approach where multiple parties train a model collaboratively without sharing raw data β each trains locally and shares only model gradients with a central aggregator. The aggregated model benefits from cross-institutional patterns. Primary risk: gradient leakage attacks can reconstruct individual records; mitigated by DP-SGD (differential privacy on gradients). SWIFT ran an experimental trial with 13 banks and Google Cloud for cross-border fraud detection over synthetic transaction data, achieving 2x effectiveness over single-institution models; the federated learning component remains pre-production as of mid-2026. Detailed in Module 11.
FF1. The NIST-standardized format-preserving encryption algorithm (SP 800-38G Rev. 1, draft). Uses a 10-round AES-based Feistel network over a configurable alphabet. Requires a minimum domain of 1,000,000 values (raised from 100 in the original 2016 standard based on small-domain attack research) β suitable for MSISDNs, IBANs, and credit card numbers, but not for small-domain fields like country codes or age values. The basis for GCP's AES-FFX mode (CryptoReplaceFfxFpeConfig). The earlier FF3-1 algorithm was removed from Rev. 1 due to vulnerabilities β FF1 is the only NIST-recommended FPE algorithm for new implementations. See FPE.
FIDA (Financial Data Access). Proposed EU regulation (COM(2023)0360) that would expand open banking to open finance β covering savings, investments, pensions, insurance, credit, and crypto assets. Part of the same legislative package as PSD3 but a separate instrument. Would be organized through Financial Data Sharing Schemes (FDSS) with industry-governed technical standards and compensation for data holders. Not yet adopted as of September 2026; if adopted, implementation expected ~2027β2028. Would widen the scope of financial data accessible via APIs, making anonymization of secondary-use data more critical. Detailed in Module 11.
Fingerprinting attack. Re-identifying an individual through their unique behavioral pattern β call sequences, location trajectories, purchase histories, browsing patterns β even when direct identifiers have been removed. As few as 4 spatio-temporal points identify 95% of individuals in mobility datasets. Also called a trajectory attack when applied to movement data. Countered by spatial/temporal generalization and differential privacy noise. Detailed in Module 12.
FINMA. Swiss Financial Market Supervisory Authority. Circular 2023/1 defines bank client data and critical data. Banking secrecy breaches (Art. 47) carry criminal penalties β up to 3 years imprisonment.
FPE. See format-preserving encryption.
Format-preserving encryption (FPE). Encrypts data while preserving the original value's length, character set, and structure β a 15-digit MSISDN encrypts to another 15-digit number. Primary value is backward compatibility with legacy systems that validate field formats. Reversible (pseudonymization, not anonymization). Standard algorithm: FF1 (NIST SP 800-38G Rev. 1). GCP supports via CryptoReplaceFfxFpeConfig.
Generalization. Reducing data granularity so multiple records share the same value β birthdate β birth year, cell tower β cell cluster, transaction amount β range bucket. The backbone of k-anonymity. Risk: over-generalize and lose utility, under-generalize and leak privacy. See generalization hierarchy.
Generalization hierarchy. A tree structure defining how field values are broadened at successive levels (e.g., Tower_A1 β Cluster_A β North District β *). Anonymization tools (ARX, GCP Sensitive Data Protection) search for the minimum generalization level that achieves the target k-anonymity threshold.
GDPR. EU General Data Protection Regulation. Broadest PII definition: any information relating to an identified or identifiable natural person. Pseudonymized data is explicitly personal data (Art. 4(5)). Anonymized data exits scope entirely (Recital 26). Fines up to 4% of global annual turnover or β¬20M.
GDPR Recital 26. Establishes the anonymization standard: data must not be identifiable "by any means reasonably likely to be used," including by third parties and future technology.
Gretel. Commercial synthetic data platform (now part of NVIDIA's ecosystem) supporting tabular, time-series, and text data generation with optional differential privacy guarantees. Provides a managed alternative to SDV for synthetic data pipelines. When used in DP mode, offers formal privacy bounds on the generated output. Requires membership inference testing regardless of DP setting β generators can memorize source records, especially outliers.
HMAC. Hash-based Message Authentication Code β a standardized keyed-hash construction (HMAC-SHA256(key, input)). Same key + same input = same output, enabling cross-table joins. Without the key, dictionary attacks are impossible. The standard production pattern for pseudonymizing finite-domain identifiers (MSISDNs, IBANs). Key managed in Cloud KMS or HSM.
Homogeneity attack. An attack on k-anonymous data where all records in an equivalence class share the same sensitive attribute value. The attacker learns that value with certainty β even without identifying the specific individual. L-diversity was designed to prevent this.
HSM. Hardware Security Module β tamper-resistant hardware that stores cryptographic keys and performs crypto operations without exposing key material. Cloud-hosted HSMs keep keys in the provider's data center; external HSMs keep them on-prem or with a third party.
IBAN. International Bank Account Number. A direct identifier. Finite domain within a national banking system makes unsalted hashes reversible.
Identity resolution. The process of mapping multiple identifiers (customer account, loyalty card, cookie/device ID, email, delivery address) to a single canonical identity. In anonymization architectures with multiple data domains (e-commerce, retail), identity resolution must happen before pseudonymization β once identifiers are pseudonymized, matching across identity systems is no longer possible. The resolved canonical ID is then pseudonymized with a shared key across all domain pipelines to enable cross-domain joins. Detailed in Module 06 (e-commerce reference architecture).
IMEI. International Mobile Equipment Identity β identifies the physical handset/device. A direct identifier.
IMSI. International Mobile Subscriber Identity β identifies the SIM/subscription on the network. A direct identifier that correlates with MSISDN.
Indonesia UU PDP. Law No. 27/2022, fully enforceable since October 2024. GDPR-inspired but does not explicitly address anonymization or pseudonymization. Implementing regulations still being finalized.
Inference attack. Using machine learning or statistical methods to deduce sensitive attribute values from quasi-identifiers β e.g., training a model on anonymized data to predict health conditions from demographic patterns. More general than the homogeneity attack (which exploits group structure) β inference attacks can succeed even when equivalence classes are diverse. Differential privacy on outputs limits the additional information an attacker gains about any individual, but cannot prevent predictions based on genuine population-level correlations between quasi-identifiers and sensitive attributes. QI reduction and generalization reduce the correlation signal available to the attacker.
InfoType. (Sensitive Data Protection) A pattern definition describing what sensitive data to scan for. GCP provides 150+ built-in InfoTypes (names, emails, credit cards, national IDs). Custom InfoTypes can be defined via regex, dictionary, or stored lists for domain-specific identifiers (MSISDNs, IMSIs).
Inspection template. (Sensitive Data Protection) A reusable, Terraform-manageable configuration specifying which InfoTypes to scan for, their likelihood thresholds, and context rules. Companion to de-identification template.
Journalist model. Adversary who wants to re-identify any individual to prove the anonymization is broken. No specific target. Risk metric: expected number of re-identifiable records. Applies to public data releases and regulatory audits.
k-anonymity. Ensures every record is indistinguishable from at least kβ1 others on its quasi-identifiers. A dataset with k=50 means every person hides in a crowd of at least 50. Minimum thresholds: k β₯ 5 baseline, k β₯ 11 for public releases. Detailed in Module 04.
KEK rotation. Rotating the key encryption key in Cloud KMS β the KEK is re-versioned and existing DEKs are re-wrapped with the new version. The DEKs themselves don't change, so pseudonyms remain stable. A routine operational task. Contrast with DEK rotation.
DEK rotation. Replacing the data encryption key used for pseudonymization transforms. Changes every pseudonym derived from the old key β records produced before and after rotation cannot be joined. For FPE/deterministic encryption, the pseudonymized zone can be re-encrypted in batch. For HMAC, rotation is irreversible (a permanent "pseudonym reset"). A data lifecycle event, not a routine operation. Detailed in Module 07.
Key rotation. See KEK rotation and DEK rotation. In anonymization architectures with envelope encryption, these are distinct operations with different impacts β KEK rotation is routine and non-disruptive, DEK rotation changes pseudonyms and breaks join continuity.
KYC (Know Your Customer). Regulatory requirements mandating financial institutions to verify the identity of their clients at onboarding and on an ongoing basis. AMLD5 Art. 13 sets a general duty to identify customers and verify their identity from reliable, independent sources; exact data elements (name, date of birth, address, nationality, ID documents) vary by national transposition and supervisory guidance. KYC data cannot be anonymized during the statutory retention period β it must remain identifiable for AML compliance. Detailed in Module 11.
Knowledge Catalog. GCP's data catalog and governance product (formerly Data Catalog, then Dataplex Universal Catalog; renamed April 2026). Manages metadata, lineage, data quality, business glossaries, and policy tags. Sensitive Data Protection publishes discovery profiles as aspects in Knowledge Catalog, making sensitivity metadata searchable across the data estate.
Lakehouse for Apache Iceberg. GCP's managed Iceberg lakehouse product (formerly BigLake; renamed April 2026). Provides a serverless Iceberg REST Catalog (Lakehouse runtime catalog), managed Iceberg tables with auto-reclustering and streaming, and cross-cloud catalog federation. BigQuery's column-level security and masking extend to Lakehouse Iceberg tables via compute delegation.
L-diversity. A privacy model extending k-anonymity: each equivalence class must contain at least l distinct, well-represented values for every sensitive attribute. Prevents the homogeneity attack. Variants: distinct (just l different values), entropy (evenly distributed), recursive. Effective when sensitive attributes have many categories; limited for binary or low-cardinality attributes. Detailed in Module 04.
LGPD. Brazil's General Data Protection Law. Mirrors GDPR closely. Includes sensitive data categories (racial origin, religious belief, health, biometric data).
Lookup key. A dedicated HMAC key used to compute subject_id_hash for DEK table lookups in a per-subject keying architecture. Separate from both the KEK and the pseudonymization DEKs, stored in KMS or Secret Manager with HSM protection. If the lookup key leaks, an attacker who knows a subject's original identifier can find their DEK table row. Detailed in Module 07.
Linkability. Whether records belonging to the same individual can be connected across or within datasets. Deterministic transformations (same input β same output) preserve linkability. Good for analytics (enables joins), bad for privacy (enables tracking).
Legitimate Interest Assessment (LIA). A documented three-step test required when relying on legitimate interest (GDPR Article 6(1)(f)) as the lawful basis for processing personal data without anonymization. Steps: (1) identify a lawful, specific, real interest, (2) assess necessity β could the goal be achieved without personal data?, (3) balancing test β does the organization's interest outweigh data subjects' rights? Pseudonymization, data minimization, and opt-out mechanisms tip the balance favorably. EDPB Opinion 28/2024 confirms legitimate interest as valid for AI model training. A one-page LIA per use case is standard. Detailed in Module 09.
Linkage attack. Cross-referencing an anonymized dataset with an auxiliary dataset on shared quasi-identifiers to re-identify individuals. The mechanism behind most real-world re-identification incidents.
Local DP. A deployment model where each data source adds noise before sending data anywhere β no central party ever sees raw individual data. Used by Apple (iOS telemetry) and Google (Chrome). Requires much more data than central DP to achieve comparable aggregate accuracy. Contrast with central DP.
Marketer model. Adversary who wants to re-identify as many individuals as commercially useful. Tolerates false positives. Risk metric: proportion of the dataset correctly re-identifiable. Applies to data broker and advertising scenarios.
Membership inference. An attack (or validation test) that determines whether a specific individual's record was in the training data of a synthetic data model. If the model memorized training records, membership inference succeeds β meaning the synthetic output contains near-real records. SDV includes evaluation metrics for this.
Merchant category code (MCC). A four-digit code classifying a merchant's business type. A quasi-identifier β combined with transaction amount and timestamp, it contributes to spending fingerprints.
Model Armor. A GCP managed service that intercepts LLM prompts and responses, screening both for PII using SDP's inspection and de-identification engine. In basic mode, uses predefined InfoTypes. In advanced mode, reuses existing SDP inspect and de-identify templates β the same custom InfoTypes and transforms used in data pipelines. Native on Agent Platform; also available as a standalone API. Detailed in Module 08.
Model inversion. An attack that reconstructs individual training records from a trained model's outputs, exploiting the model's memorization of training data. Most effective when the attacker has query access to model predictions with confidence scores. Countered by DP training, output perturbation, and limiting confidence-score granularity. Related to membership inference but more severe β model inversion recovers actual data, not just presence/absence. Detailed in Module 12.
MiFID II (Markets in Financial Instruments Directive). EU directive governing investment services and financial markets. Requires 5-year retention of transaction records for regulatory reporting β anonymization of reportable transactions is prohibited during the retention period. Relevant to anonymization zone design in financial services: MiFID-mandated data stays in Zones 0β1. Detailed in Module 11.
MSISDN. Mobile Station International Subscriber Directory Number β the subscriber's phone number in E.164 format (~15 digits). A direct identifier. Finite domain makes unsalted hashes trivially reversible.
Network tokenization. Tokenization of payment card numbers (PANs) performed by the card network (Visa VTS, Mastercard MDES) rather than the merchant. Network tokens are bound to a token domain/requestor: card-on-file tokens are generally merchant-specific, while device/wallet tokens (Apple Pay, Google Pay) can be accepted across merchants. Tokens auto-update on card reissue and may qualify for lower interchange fees. Systems handling only network tokens can be out of PCI-DSS scope, provided the tokenization system is properly segmented and the token-handling systems cannot retrieve PANs or affect the CDE. Contrast with merchant-generated tokens, which are specific to one payment processor. Detailed in Module 11.
NILM (Non-Intrusive Load Monitoring). The inference of individual appliance usage from aggregate electricity consumption data. At 15-minute smart meter resolution, NILM can detect appliance types (oven, EV charger, medical equipment), occupancy patterns, and household routines. Research shows 5 consecutive half-hourly readings can re-identify over 90% of households. The energy equivalent of the "4 spatio-temporal points" finding for mobility data. Detailed in Module 11.
NIST SP 800-38G. The NIST standard specifying format-preserving encryption. Rev. 1 (2025 draft) retains FF1 only; FF3-1 was removed due to vulnerabilities over small domain sizes.
NIST SP 800-57. NIST Special Publication 800-57 Part 1 Rev 5 β the primary reference for cryptographic key management recommendations. Sets maximum cryptoperiods (2 years for symmetric encryption keys, 2 years for HMAC keys) and defines key lifecycle states. Referenced in Module 10 for pseudonymization key rotation guidance.
Non-identifying field. A field that carries no re-identification risk and requires no treatment. Examples: record type, network technology, billing plan category.
OpenDP. Open-source DP framework (MIT, Harvard Privacy Tools project) offering composable privacy primitives with formal verification β each operation carries a machine-checkable proof of its privacy guarantee. For use cases requiring auditable DP guarantees or fine-grained composition control.
OpenLineage. An open standard for data lineage metadata, defining a common format for lineage events (run, job, dataset, facets). GCP's Data Lineage API accepts OpenLineage-formatted events via ProcessOpenLineageRunEvent, enabling custom anonymization-specific lineage attributes (transform type, SDP template version, KMS key version) alongside automatic lineage. Detailed in Module 10.
PAN (Primary Account Number). The card number on a credit or debit card β typically 16 digits, with the first 6β8 forming the BIN (issuing bank identifier). A direct identifier. PCI-DSS Req. 3.5.1 requires PANs to be rendered unreadable wherever stored β via tokenization, truncation, keyed cryptographic hashes of the entire PAN (Req. 3.5.1.1), or strong encryption. Req. 3.4.1 requires masking when displayed. Finite domain within a BIN range makes unkeyed hashes reversible. See PCI-DSS, network tokenization.
PCI-DSS (Payment Card Industry Data Security Standard). Security standard governing the protection of cardholder data β primarily the PAN. Version 4.0.1 (published June 2024, sole active version from January 2025; future-dated requirements mandatory March 31, 2025) requires PANs to be rendered unreadable when stored (Req. 3.5.1) via tokenization, truncation, keyed cryptographic hashes of the entire PAN (Req. 3.5.1.1), or strong encryption, and masked when displayed (Req. 3.4.1). Tokenization is the primary mechanism for scope reduction β but each component must be individually assessed to confirm it cannot retrieve PANs or access or affect the tokenization system/CDE before it may be excluded from scope. Key management must follow defined cryptoperiods (Req. 3.7.4). Relevant beyond payments: the same vault and key management patterns apply to any high-sensitivity identifier. Detailed in Modules 03, 07, and 11.
Per-subject keying. An architecture where each data subject has their own DEK for pseudonymization, enabling individual crypto-shredding (destroy the subject's DEK to erase their pseudonyms). Detailed in Module 07.
Perturbation. Adding controlled random noise to data values. Individual records become unreliable; aggregate statistics remain accurate (symmetric noise cancels out over groups). The core mechanism behind differential privacy. Spatial perturbation randomly shifts GPS coordinates within a defined radius.
PipelineDP. Google's open-source DP library (Python and JVM) providing end-to-end differentially private aggregations with native Apache Spark and Beam integration. The Spark-native alternative to Privacy on Beam.
PLD (Privacy Loss Distribution). A design-time tool for computing tight composition bounds by tracking the full privacy-loss distribution across queries. Answers "given our target total Ξ΅ and the number of reports we need, what per-query Ξ΅ should we use?" Can often unlock significantly more queries for the same total epsilon compared to naive budget accounting (typically 20β40%, depending on mechanisms and parameters).
Policy tag. (BigQuery) A label attached to a column that controls access and masking behavior. Combined with masking rules, enables dynamic data masking β Fine-Grained Reader sees raw data, Masked Reader sees the masked output.
Post-processing immunity. A property of differential privacy: any computation applied to the output of a DP mechanism preserves the DP guarantee. Filter, transform, join, train a model β the result is still DP-protected. Privacy is enforced once at query time and holds regardless of what consumers do downstream. Contrast with k-anonymity, where joining with auxiliary data can destroy the guarantee.
Presidio. Microsoft's open-source toolkit (Apache 2.0) for PII detection and anonymization. Combines regex, rule-based logic, and NLP-based named entity recognition (spaCy NER). Stronger than Sensitive Data Protection for unstructured text. Presidio-Structured extends detection to PySpark DataFrames. Self-hosted (Docker, Kubernetes, embedded Python).
Privacy budget. The cumulative Ξ΅ consumed across all DP queries over a dataset. Once exhausted, further queries violate the privacy guarantee. Budget management β allocation, tracking, and reset strategy β is a key architectural decision. Detailed in Module 04.
Privacy on Beam. Google's open-source end-to-end DP framework built on Apache Beam. Wraps Beam's aggregation primitives with DP-aware versions that automatically add calibrated noise. Tracks cumulative privacy budget across aggregations. Runs on Dataflow, Flink, or Spark.
Privacy-utility tradeoff. The fundamental tension: stronger privacy requires more aggressive data transformation, which reduces analytical value. Every anonymization technique sits somewhere on this curve.
Prosecutor model. Adversary who knows a specific individual is in the dataset and wants to find their record. Strongest adversary; sets the strictest privacy bar. Risk metric: probability of correctly identifying the target.
PSD2 (Payment Services Directive 2). EU directive introducing third-party access to customer bank account data via APIs β Account Information Services (AIS) and Payment Initiation Services (PIS). Requires explicit, granular, revocable customer consent. Data shared via PSD2 APIs is personal data under GDPR β anonymization applies to aggregate analytics derived from that data, not to the operational API flow. Succeeded by PSD3 (political agreement November 2025, expected ~2027β2028), which reforms payment services and open banking. The separate proposed FIDA regulation (COM(2023)0360), not yet adopted, would extend data access to broader financial products. Detailed in Module 11.
Pseudonymization. Replacing direct identifiers with substitutes (tokens, hashes, encrypted values) while retaining the ability to re-link records to individuals via a key, mapping table, or vault. The data remains personal data under GDPR. If any reversal path exists, the data is pseudonymized, not anonymized.
Pseudonymisation domain. (EDPB Guidelines 01/2025) The context within which pseudonymisation prevents attribution to a specific data subject. Controllers must define the domain's scope (organizational unit, specific external recipients) and ensure re-identification information is kept outside it. Pseudonymisation within a domain supports the LIA balancing test but does not exit GDPR scope. Detailed in Module 09.
Quasi-identifier. A field that does not identify alone but can in combination with other fields. Examples: ZIP code, date of birth, cell tower ID, transaction timestamp, merchant category, device fingerprint. Primary re-identification risk β 4 data points can identify 90β95% of individuals in many datasets.
RACI matrix. A responsibility assignment framework: Responsible (does the work), Accountable (owns the outcome), Consulted (provides input), Informed (kept in the loop). In Module 09, used to assign ownership across anonymization decisions β classification policy, technique selection, key rotation, incident response, external sharing approval, and DPIA execution.
Ranger. See Apache Ranger.
Reconstruction attack. Rebuilding individual-level records from aggregate statistics using optimization or constraint solving. Given enough overlapping aggregates (counts, sums, averages), an attacker can solve for individual values. The 2020 US Census adopted differential privacy specifically to counter this attack class. Countered by differential privacy with budget tracking. Detailed in Module 12.
Referential consistency. (SDM) Applying the same masking transformation to the same value across all tables in a dataset copy β e.g., customer ID "CUST-0042" becomes "CUST-7819" in every table. Without this, foreign key relationships break and the masked dataset is unusable for testing.
Re-identification. Linking a record in an anonymized or pseudonymized dataset back to the individual it represents. Achieved through quasi-identifiers, auxiliary datasets, or brute-force attacks on weak transformations.
Re-identification event. An incident where anonymized data is matched to a specific individual β through a deliberate attack, accidental discovery, or external assessment. Triggers a structured response: detection, containment (first 4 hours), assessment (first 24 hours), remediation. May qualify as a GDPR personal data breach requiring notification under Articles 33/34. Detailed in Module 10.
Re-identification risk. The probability that an attacker successfully re-identifies an individual. Measured by record uniqueness on quasi-identifier combinations. Tools like ARX quantify this automatically.
Reversibility. Whether the original value can be recovered from the transformed output. If any reversal path exists (key, mapping table, vault), the data is pseudonymized, not anonymized.
ROPA (Records of Processing Activities). A document required by GDPR Article 30 for most data controllers. Article 30 mandates recording: categories of data, purposes, recipients, transfers, retention periods, and security measures. For anonymization programs, it is recommended to additionally document: the anonymization technique applied, privacy model and parameters (k value, DP epsilon), and the justification for classifying output as anonymous vs pseudonymous β these are not statutory requirements but strengthen audit readiness. Typically the first document a DPA requests in an inquiry. Detailed in Module 10.
Row-level access policy. (BigQuery) A filter expression that evaluates per row and per querying principal, transparently restricting which rows a user can see. Complements column-level masking (DDM) β together they provide "which columns" and "which rows" access control on a single table without creating separate physical copies. Detailed in Module 06.
SAR (Suspicious Activity Report). A report filed with a Financial Intelligence Unit when a financial institution detects potentially suspicious transactions or behavior. Must include full identifying information about the parties involved. Subject to "no tipping off" rules β the institution must not inform the data subject that a report has been filed, which restricts standard GDPR data subject notification (resolution: GDPR Art. 23). Cannot be anonymized during retention. Detailed in Module 11.
SBA Cloud Guidelines. Non-binding, risk-based guidance from the Swiss Bankers Association (3rd edition, November 2025) on how anonymization, pseudonymization, and encryption interact with banking secrecy for cloud deployments. Provider inability to re-identify is one important control but does not by itself satisfy all outsourcing and secrecy obligations β broader contractual, organizational, and audit measures also apply.
SCHUFA ruling. CJEU decision (December 2023) holding that automated credit scoring can constitute automated decision-making under GDPR Article 22 when a lender gives the score a determining role in granting, performing, or terminating a contract. In that scenario, the credit reference agency is subject to Article 22 transparency, human intervention, and contestation requirements. Creates dual compliance requirements with the EU AI Act (which classifies credit scoring as high-risk AI). Detailed in Module 11.
Schema drift. An unplanned change to the structure of incoming data β a new column, a renamed field, a changed data type. In anonymization pipelines, schema drift is the most common cause of silent failure: a new PII column bypasses transforms because the SDP de-identification template has no rule for it. The defense is a schema validation step that halts the pipeline on unknown fields rather than passing them through raw. Detailed in Module 10.
sdcMicro. R package for statistical disclosure control β k-anonymity, local suppression, post-randomization (PRAM), microaggregation, and formal risk measures. Standard in the European statistical community. Useful when clients need risk quantification reports that auditors and DPAs recognize.
SDM. See static data masking.
SDV (Synthetic Data Vault). The most widely used open-source library for synthetic data generation (Python). Supports tabular, relational (multi-table), and time-series data. Includes evaluation metrics for membership inference testing. No native DP mode β for DP-guaranteed synthesis, see DPSynth.
Secure multi-party computation (SMPC). A cryptographic protocol where multiple parties jointly compute a function over their combined inputs without revealing any party's individual data. Computationally expensive but provides strong mathematical guarantees. Practical for narrow, high-value queries (e.g., "has this account appeared in suspicious activity reports at other banks?") β not for bulk analytics. Used in cross-institutional fraud detection and AML compliance. Detailed in Module 11.
Selective masking. Masking only the fields irrelevant to the LLM task while leaving task-relevant fields intact β e.g., redacting names and account numbers but preserving complaint text for sentiment analysis. A risk-based decision requiring agreement between data engineering, privacy, and AI teams on which fields carry privacy risk vs. task value. Detailed in Module 08.
Sensitive attribute. The value an attacker wants to learn: diagnosis, salary, credit score, purchase details, call content. The target of an attack, not the mechanism.
Sensitive Data Protection (SDP). GCP's primary service for PII detection and de-identification (formerly Cloud DLP). Combines inspection (150+ built-in InfoTypes, custom detectors) with de-identification transforms (FPE, deterministic encryption, keyed hashing, masking, bucketing, redaction). Includes a discovery service for continuous profiling. Detailed in Module 05.
Sensitivity (DP). How much a single record can affect an aggregate output. Determines how much noise differential privacy must add. A field ranging 0β3,600 (call duration in seconds) has higher sensitivity than a binary field (answered/unanswered).
Sensitivity taxonomy. A hierarchical classification scheme mapping data types to sensitivity tiers (e.g., Critical / High / Medium / Low / Public). Each tier defines both the anonymization requirement (must pseudonymize, may generalize, no treatment needed) and the access scope (break-glass only, masked for analysts, broad internal). Drives technique selection and BigQuery policy tag assignment. Defined in the anonymization policy. Detailed in Module 09.
Singapore PDPA. Personal Data Protection Act. Anonymized data exits scope. Notable: unauthorized re-identification is a criminal offense (up to 2 years imprisonment). APPA Anonymization Guide (June 2025) provides a five-step process.
Smart Meter Gateway (SMGW). A BSI-certified secure communication gateway (BSI TR-03109) with a dedicated security module, required in Germany as part of an intelligent metering system in applicable rollout cases β not mandated for every digital meter. Mediates all data flows between the meter and external parties, enforcing purpose- and role-specific data access: grid operators receive 15-minute interval data, energy suppliers receive daily or monthly totals for standard billing (with quarter-hourly access for dynamic tariff customers), consumers get real-time access. An example of privacy-by-design enforced at the infrastructure layer. Detailed in Module 11.
Static data masking (SDM). A strategy that permanently transforms a copy of production data for dev/test/staging use. Applies per-field transformations (substitution, suppression, generalization, date shifting, shuffling, FPE) with referential consistency across tables. Irreversible by design β no mapping back to originals. Contrast with dynamic data masking.
Structural anonymization. Transforming a dataset as a whole β grouping records, generalizing quasi-identifiers, suppressing outliers β to satisfy formal privacy models (k-anonymity, l-diversity, t-closeness). Operates at the dataset level, not per-record. Fundamentally a batch operation. Contrast with per-record pseudonymization.
Suppression. Removing data entirely β the simplest privacy technique. Appropriate for fields with no analytical value (free-text notes, unused direct identifiers) or outlier records that cannot be generalized.
Synthetic data. Entirely artificial records generated by statistical models trained on real data. Preserves distributions and correlations but contains no real individuals. Exits GDPR scope if the generation process does not memorize training records. See SDV, membership inference, DP-SGD.
T-closeness. The strongest model in the k-anonymity family. Requires that the distribution of sensitive attribute values within each equivalence class be close to the overall dataset distribution, measured by Earth Mover's Distance (EMD). Prevents attribute inference from distribution skew. Severe utility loss and hard to parameterize β in practice, most systems use k-anonymity + DP instead. Detailed in Module 04.
Taxonomy (BigQuery). A named classification scheme (e.g., PII_Classification) scoped to a project and region. Contains policy tags that attach to columns and control masking behavior. Common designs: sensitivity-based (High/Medium/Low), role-based, or regulation-based. Terraform-manageable via google_data_catalog_taxonomy. The organizing structure for BigQuery dynamic data masking. Detailed in Module 06.
Token vault. A secure database mapping tokens to original values. The single point of truth for detokenization β and the single point of compromise. Vault-based tokenization offers maximum security but adds latency and availability challenges at high throughput.
Tokenization. Replacing a sensitive value with a random token that has no mathematical relationship to the original. Unlike hashing, tokens cannot be reversed by enumerating the input space β without vault access, they are meaningless. Canonical use case: PCI-DSS scope reduction. Two designs: vaulted (mapping stored in a token vault) and vaultless (token derived via FPE, eliminating vault dependency).
VPC Service Controls (VPC-SC). A GCP network-level perimeter that prevents data from leaving a defined set of projects β even by principals with IAM permissions. In anonymization architectures, VPC-SC wraps the raw and pseudonymized zones to prevent data exfiltration. Acts as defense-in-depth that catches misconfigured IAM. The structurally anonymized zone may sit inside or outside the perimeter depending on sensitivity; the DP-guaranteed zone is typically outside, since its mathematical guarantee makes network containment unnecessary. Detailed in Module 06.
Created by Claude Opus with β€οΈ for the Aliz Data Engineering team.