3. Techniques
This module covers every major anonymization and pseudonymization technique you'll encounter on client engagements. For each technique, it explains the mechanism, when to use it, what can go wrong, and how it fits into a layered architecture. These are the building blocks — Module 04 adds the formal privacy models that provide mathematical guarantees, and Module 06 shows how techniques combine in production pipelines.
Suppression and redaction
Suppression is the simplest privacy technique: remove the data entirely. No value exists in the output, so no value can be attacked. Redaction is the variant that replaces values with a placeholder (null, ***, [REDACTED]) rather than dropping the field altogether.
When to remove data entirely
Suppression is the right choice when a field has no analytical value for the intended use case and carries privacy risk. Common candidates:
- Free-text fields containing names, addresses, or personal notes — customer support notes in telco, delivery instructions in e-commerce, loan officer comments in fintech
- Fields that are pure direct identifiers with no downstream use — full name, national ID, email address in datasets where these aren't needed for the analysis
- Outlier records that cannot be generalized without creating a singleton equivalence class (a concept from k-anonymity — see Module 04). If one record has a unique combination of quasi-identifiers even after generalization, suppressing that record may be preferable to distorting the entire dataset
Suppression is not a technique to reach for by default. It is the nuclear option — what you use when the data's privacy risk outweighs its analytical value, or when no other technique can adequately treat a field.
Static vs dynamic redaction
Static redaction permanently removes or replaces values in a copy of the dataset. The transformation is applied once, and the output contains no trace of the original value. This is appropriate for datasets that will leave controlled environments — shared with partners, used in non-production systems, or published.
Dynamic redaction masks values at query time without modifying stored data. The underlying value remains intact; the masking is applied by the query engine based on the caller's identity and permissions. In BigQuery, this is implemented via policy tags and data masking rules (Module 05). In Hadoop ecosystems, Apache Ranger provides equivalent functionality.
Dynamic redaction is not an anonymization technique — it is an access control mechanism. The data remains fully identifiable at rest. It belongs in the pseudonymization layer of a data zones architecture (Module 06), not at the anonymization boundary.
When a client says "we redact the sensitive fields," ask: static or dynamic? If dynamic, the underlying data is still personal data with full regulatory obligations. If static, ask what happens to the quasi-identifiers — redacting only direct identifiers while leaving behavioral patterns intact is the most consistently broken approach in the field.
Generalization and aggregation
Generalization reduces data granularity so that multiple records share the same attribute values, creating what privacy literature calls equivalence classes — groups of records that are indistinguishable on a given set of fields.
Reducing granularity to create equivalence classes
The mechanism is straightforward: replace specific values with broader categories. A birthdate becomes a birth year. An exact address becomes a district. A GPS coordinate snaps to a grid square. A transaction amount maps to a range bucket.
The goal is to ensure that no individual can be singled out based on their combination of quasi-identifier values. If every record in the dataset shares its age range, district, and gender with at least k−1 other records, an attacker who knows a target's demographics cannot isolate their record. This is the intuition behind k-anonymity (Module 04).
Common patterns
| Original value | Generalized value | Context |
|---|---|---|
| Birthdate (1987-03-15) | Birth year (1987) or age band (35–39) | All verticals |
| Street address | District or postal code prefix | Retail, fintech, energy |
| GPS coordinate | Grid square (100m², 500m², 1km²) | Telco location data, energy smart meters |
| Cell tower ID | Cell cluster (group of 5–10 towers) | Telco CDRs |
| Timestamp (14:23:07) | Hour bucket (14:00–15:00) | All verticals |
| Transaction amount (€47.83) | Range bucket (€25–€50) | Fintech, retail |
| Call duration (3m42s) | Category (1–5 min) | Telco CDRs |
| Product SKU | Product category | E-commerce |
| Merchant name | Merchant category code (MCC) | Fintech |
| Smart meter reading (15-min) | Hourly or daily aggregate | Energy/utilities |
Aggregation takes this further by collapsing individual records into group-level statistics. Instead of individual CDRs, you output "cell cluster X had 1,247 calls between 14:00–15:00." This is stronger privacy — individual records no longer exist — but it eliminates record-level analysis entirely.
Risks
Over-generalization destroys analytical utility. If you generalize timestamps to daily buckets, you lose the ability to analyze peak-hour patterns. If you generalize cell towers to regional clusters, network planning becomes impossible. The engineering challenge is finding the generalization hierarchy that meets the privacy threshold (typically a k-anonymity target) while preserving the analytical resolution the use case requires.
Under-generalization leaves individuals identifiable. Location data is particularly resistant to generalization — research shows that even coarsened spatio-temporal data retains high re-identification potential (de Montjoye et al., 2013). A grid square that contains only one subscriber's home address at 3 AM is effectively a direct identifier, regardless of how many people pass through during rush hour.
Generalization hierarchies
Production anonymization tools (ARX, GCP Sensitive Data Protection) use configurable generalization hierarchies — tree structures that define how values can be broadened at successive levels. For a cell tower field:
Level 0 (original): Tower_A1, Tower_A2, Tower_A3, Tower_B1, Tower_B2, ...
Level 1 (cluster): Cluster_A, Cluster_A, Cluster_A, Cluster_B, Cluster_B, ...
Level 2 (district): North, North, North, South, South, ...
Level 3 (suppress): *, *, *, *, *, ...
The anonymization algorithm searches for the minimum generalization level that achieves the target k-anonymity threshold. Different fields can be generalized to different levels — a timestamp might need only Level 1 (hourly bucket) while a cell tower needs Level 2 (district) to reach k=50. The tool optimizes for information loss across all fields simultaneously.
Defining these hierarchies is a significant piece of work in a client engagement — they encode domain knowledge about which groupings are meaningful. A cell tower hierarchy should reflect geographic proximity, not arbitrary ordering.
Hashing
A cryptographic hash function maps an input to a fixed-length digest (e.g., SHA-256 produces a 256-bit output). The same input always produces the same hash, the function is computationally irreversible (you cannot derive the input from the output), and different inputs should produce different outputs.
This makes hashing appealing for pseudonymization: replace an MSISDN (phone number) with its SHA-256 hash, and the value looks opaque but remains consistent across tables — enabling joins and aggregations without exposing the original.
Plain hashing and why it fails for finite domains
The critical problem: for inputs drawn from a small or enumerable space, an attacker can precompute the hash of every possible value and look up any hash in the dataset. This is called a dictionary attack (or rainbow table attack for precomputed lookup tables).
Telco identifiers are acutely vulnerable. An MSISDN follows the E.164 international format — country code + subscriber number, typically 10–15 digits total. A national numbering plan might contain 100 million valid phone numbers. Hashing every one with SHA-256 takes minutes on commodity hardware. The same vulnerability applies to any finite-domain input:
| Identifier | Domain size | Time to enumerate (single GPU) |
|---|---|---|
| MSISDN (national) | ~10⁸ | Minutes |
| IBAN (country-specific) | ~10⁹–10¹⁰ | Hours |
| Credit card number (BIN-constrained) | ~10⁶ per BIN | Seconds |
| IPv4 address | ~4.3 × 10⁹ | Hours |
| Social security number (US) | ~10⁹ | Minutes |
Plain hashing of finite-domain identifiers is not a privacy technique — it is a transformation that provides a false sense of security. The NYC taxi data incident is the textbook case: medallion numbers were hashed with MD5 without a salt, and the entire mapping was reversed by hashing all possible medallion numbers.
Salted hashing and HMAC
Adding randomness or a secret key to the hash input prevents straightforward precomputation. However, the security properties differ significantly depending on whether the added value is a salt (typically not secret — stored alongside the hash) or a cryptographic key (secret — stored in KMS).
Two approaches:
Salted hashing (SHA-256(MSISDN || salt)) — prepends or appends a random value to the input before hashing. Different salts produce different outputs for the same input. If each dataset uses a different salt, cross-dataset joins are impossible — which strengthens privacy but eliminates linkability.
HMAC (HMAC-SHA256(key, MSISDN)) — a standardized keyed-hash construction that provides stronger security guarantees than naive salting. The key is a cryptographic secret managed separately from the data. Crucially, the same key produces consistent pseudonyms — the same MSISDN always maps to the same hash — which preserves join capability across tables that share the key.
| Property | Plain SHA-256 | Salted SHA-256 | HMAC-SHA256 |
|---|---|---|---|
| Dictionary attack resistant | No | Prevents precomputed rainbow tables, but if the salt is known (standard practice), finite domains are still enumerable per salt | Yes — key must remain secret |
| Cross-dataset joins | Yes | No (different salts) | Yes (same key) |
| Consistency (same input → same output) | Yes | Per-salt | Per-key |
| Key management required | No | Per-salt | Yes — centralized |
| Recommended for production | No | Situational | Yes |
For analytics pipelines that need to join pseudonymized records across tables — customer activity across billing, network, and support systems (telco); transactions across accounts and merchants (fintech); orders across inventory and delivery (e-commerce) — HMAC with a centralized key in Cloud KMS or HSM is the standard production pattern. Module 05 covers the GCP implementation; Module 07 covers key architecture and rotation.
Key rotation for right-to-erasure support
GDPR Article 17 gives individuals the right to request erasure of their personal data. In a pseudonymized dataset, deleting a specific individual's records requires knowing which pseudonym corresponds to that individual — which means maintaining a reverse-mapping or re-running the hash with the individual's identifier to find matching records.
An alternative approach: key rotation. When the HMAC key is rotated, all existing pseudonyms become meaningless — they cannot be linked to the original identifiers or to pseudonyms generated with the new key. This effectively achieves erasure at the dataset level without deleting individual records. The old pseudonymized data is retained (it contains no linkable information), while new data is pseudonymized with the fresh key.
This is a powerful pattern but involves tradeoffs: key rotation breaks all cross-table joins that depend on pseudonym consistency. It is best suited for use cases where periodic "linkability reset" is acceptable — for example, rotating keys monthly for datasets used in aggregate reporting, while maintaining stable pseudonyms for operational joins in a separate zone.
When a client says "we hash the phone numbers," the immediate follow-up is: plain or keyed? If plain, it's equivalent to cleartext for any attacker who can enumerate the national numbering plan. Recommend HMAC-SHA256 with keys managed in Cloud KMS as the minimum viable pseudonymization for any finite-domain identifier.
Tokenization
Tokenization replaces a sensitive value with a random or algorithmically derived token that has no mathematical relationship to the original. Unlike hashing, where the output is derived from the input through a deterministic function, a token is generated independently — without access to the token system, there is no path from token to original value.
The canonical use case is PCI-DSS scope reduction for payment card numbers, but the pattern applies directly to any high-sensitivity identifier: MSISDNs, IMSIs, IBANs, account numbers, national IDs.
Vaulted vs vaultless designs
Vaulted tokenization maintains a secure database (the token vault) that maps original values to tokens. The vault is the single point of truth for reversing the transformation — and the single point of compromise. To detokenize, an authorized process queries the vault with the token and receives the original value.
Vaultless (FPE-based) tokenization derives tokens cryptographically — typically using format-preserving encryption (see next section) — eliminating the vault dependency. The token is generated deterministically from the original value and a key, so re-tokenization of the same input produces the same token without vault lookup. This eliminates vault latency and availability concerns but reintroduces key management as the security dependency.
| Property | Vaulted | Vaultless (FPE-based) |
|---|---|---|
| Security model | Vault compromise = full exposure | Key compromise = full exposure |
| Latency | Vault lookup per operation | Crypto operation (fast) |
| Scalability | Vault becomes bottleneck at scale | Scales with compute |
| Reversibility | Yes — via vault | Yes — via key |
| Join capability | Consistent tokens across systems sharing vault access | Consistent tokens across systems sharing key |
| Infrastructure dependency | High-availability vault (database) | Key management (KMS/HSM) |
Scale considerations for high-throughput pipelines
At telco CDR volumes — millions to billions of records per day — vault-based tokenization introduces significant architectural challenges:
- Latency: Every record requires a vault lookup (or batch query). Even with connection pooling and caching, this adds milliseconds per record that compound across high-throughput streaming pipelines.
- Availability: The vault becomes a critical infrastructure dependency. If the vault is down, the pipeline stalls. This requires vault replication, failover, and careful capacity planning.
- Cache consistency: Caching token mappings at the pipeline layer improves throughput but introduces a window where the cache and vault can diverge — particularly during key rotation or token revocation.
For fintech payment processing, PCI-DSS already mandates tokenization infrastructure, so the vault exists. For telco and e-commerce use cases where the primary goal is pseudonymization (not PCI compliance), HMAC-keyed hashing often provides equivalent privacy guarantees with simpler infrastructure — the trade-off is that HMAC is irreversible while tokenization supports detokenization.
Format-preserving encryption (FPE)
Format-preserving encryption encrypts data while maintaining the original value's length, character set, and structure. A 15-digit MSISDN encrypts to another 15-digit numeric string. An IPv4 address encrypts to another valid IPv4 address. A 16-digit credit card number encrypts to another 16-digit number — though note that FPE preserves length and character set only, not checksums like Luhn or IBAN check digits. Identifiers with embedded checksums require separate checksum recalculation after encryption if downstream systems validate them.
FF1
The NIST-standardized FPE algorithm is FF1 (NIST SP 800-38G Rev. 1, draft), an AES-based Feistel network operating over a configurable alphabet:
| Property | FF1 |
|---|---|
| Rounds | 10 |
| Minimum domain size | 1,000,000 values (Rev. 1 draft; raised from 100 in the original 2016 standard) |
| Tweak size | Arbitrary length |
| GCP support | CryptoReplaceFfxFpeConfig (AES-FFX mode) |
FF1 is the basis for GCP Sensitive Data Protection's FPE implementation. The Rev. 1 draft raised the minimum domain size from 100 to 1,000,000 based on cryptanalytic research showing small-domain FPE is vulnerable to attacks. This means FF1 is suitable for fields with large numeric domains like MSISDNs (~10 billion possible values in E.164) — but not for small-domain fields like country codes, short category codes, or two-digit age values. For those, use generalization or bucketing instead. For checksum-bearing identifiers (credit card numbers with Luhn, IBANs with check digits), FPE preserves length and character set but not the checksum — downstream validation will fail unless the checksum is recalculated post-encryption.
An earlier algorithm, FF3-1, was ~2× faster but had significant vulnerabilities when used over small domain sizes. The original FF3 was withdrawn after a cryptanalytic attack, and FF3-1 (the patched version) was subsequently removed from NIST SP 800-38G Rev. 1 (2025 draft), which retains FF1 only. Legacy systems may still reference FF3-1, but FF1 is the only FPE algorithm retained in the current Rev. 1 draft. The 2016 SP 800-38G remains the published standard until Rev. 1 is finalized, but FF1 is the safe choice for new implementations regardless.
Why legacy systems love FPE
FPE's primary value is backward compatibility. An MSISDN field has length constraints, character-type validations, and format checks baked into application logic, database schemas, and API contracts across billing, mediation, CRM, and analytics systems. Replacing it with a UUID or 64-character hash triggers cascading changes across the entire estate. FPE sidesteps this — the encrypted MSISDN looks like a phone number, passes validation, and flows through every downstream system unchanged.
GCP Sensitive Data Protection supports FPE via CryptoReplaceFfxFpeConfig (AES-FFX mode), with the encryption key managed in Cloud KMS (Module 05).
Pseudonymization, not anonymization
FPE is reversible — anyone with the encryption key can decrypt and recover the original value. This makes it a pseudonymization technique, not anonymization, regardless of how the encrypted output looks. The data remains personal data under GDPR, with full regulatory obligations.
This distinction matters in client conversations. FPE is sometimes described as "anonymization" because the output is visually unrecognizable, but the legal and technical classification depends on reversibility, not appearance.
Risks and limitations
- Format leakage. Preserving format can leak information — an encrypted MSISDN starting with
+36still reveals the country. Always encrypt the entire value, not just a portion. - Deterministic output. Same key + same input = same ciphertext. This preserves join capability but also linkability — the same encrypted MSISDN across datasets identifies the same subscriber without decryption.
- Small-domain vulnerability. For domains with few possible values (e.g., a 3-digit code), an attacker with encryption oracle access can encrypt all values and build a complete mapping.
FPE is the right choice when format compatibility is a hard constraint — legacy system integration, schema migration avoidance, or phased rollouts where some systems still expect original-format data. For greenfield architectures with no format constraints, HMAC-keyed hashing is simpler and avoids the reversibility surface.
Data masking
Data masking transforms data to obscure sensitive values. Unlike hashing or encryption, masking techniques are typically designed for human readability — replacing characters with asterisks, showing only the last four digits, or substituting with realistic-looking fake values.
Static data masking (SDM) for non-production environments
Static data masking permanently transforms a copy of production data for use in development, testing, and staging environments. The transformation is applied once during the copy process, and the output dataset contains no original sensitive values.
SDM is a strategy, not a single technique. It applies per-field transformations chosen from the techniques covered in this module — realistic substitution (fake but plausible values), suppression, generalization, date shifting, shuffling (swapping values between records), and sometimes FPE or hashing. What makes SDM distinct from applying these techniques individually:
- Scope: SDM transforms an entire dataset copy as a unit, not just selected fields in a production pipeline.
- Referential consistency: If a customer ID appears in multiple tables, the same transformation is applied everywhere — preserving foreign key relationships and join correctness across the masked copy. Without this, the dataset is structurally broken for testing.
- Purpose: The output is a realistic-looking but de-identified copy for dev/test use — it must preserve schema structure, data types, value distributions, and referential integrity while containing no real individuals.
- Irreversibility: Unlike tokenization (which maintains a vault for reversal), SDM is typically irreversible by design. There is no mapping back to originals — that's the point.
Common per-field SDM transformations:
| Field type | Masking approach | Example |
|---|---|---|
| Name | Realistic substitution | "Maria Kovács" → "Anna Tóth" |
| Domain-preserving substitution | maria@example.com → user4821@example.com | |
| Phone number | Random replacement or FPE | "+36 30 123 4567" → "+36 30 847 9213" |
| Address | Street/number randomization, postal code preserved | "Váci út 33" → "Andrássy út 78" |
| Date of birth | Date shifting (± random days) | "1987-03-15" → "1987-06-22" |
| Account balance | Proportional noise or range bucketing | "€12,847.33" → "€13,100.00" |
| Foreign key (customer ID) | Consistent substitution across all tables | "CUST-0042" → "CUST-7819" (same everywhere) |
SDM tools (Delphix, Informatica, or custom Dataflow/Spark pipelines) typically take a masking policy as input — a mapping of field names to transformation types — and apply it consistently across all tables in a single run.
Dynamic data masking (DDM) as an access-control mechanism
Dynamic data masking applies transformations at query time based on the caller's identity and role. The stored data is never modified — different users see different views of the same table.
In BigQuery, DDM is implemented through policy tags and data masking rules:
- A column is tagged with a policy tag (e.g.,
PII: High) - A masking rule is attached to the policy tag (e.g.,
SHA256,Nullify,Default Value,Last 4 Characters) - Users with the
Fine-Grained Readerrole see raw data; users with theMasked Readerrole see the masked output
This is the right mechanism for controlling access to pseudonymized data within an organization — data scientists who need full detail get one view, business analysts who need only patterns get another. But DDM is not a data transformation and does not produce anonymized output. The data at rest remains fully identifiable.
DDM vs SDM — choosing correctly
| Criterion | Static masking (SDM) | Dynamic masking (DDM) |
|---|---|---|
| Data at rest | Permanently transformed | Original, unchanged |
| Use case | Non-production environments, data sharing | Role-based access control within production |
| Reversibility | Irreversible (the copy has no original values) | N/A — original data is never modified |
| GDPR status of output | Depends on technique (may be anonymized or pseudonymized) | Still personal data — same data, different views |
| Performance impact | None (transformation is pre-applied) | Query-time overhead (typically negligible) |
| When to use | Data leaving controlled environment | Data staying in controlled environment with multiple access levels |
A common pattern in data zones architectures (Module 06): DDM on the pseudonymized zone (Zone 1) for internal access control, SDM for copies exported to non-production environments or partners.
Perturbation and noise injection
Perturbation adds controlled randomness to data values, making individual records unreliable while preserving population-level statistical properties. It is the core mechanism behind differential privacy (Module 04), but it can also be applied as a standalone technique.
Calibrated noise for numerical values
For numerical fields — call duration, transaction amount, energy consumption, page view count — perturbation adds random noise drawn from a defined distribution (typically Laplace or Gaussian). The noise is calibrated so that:
- Individual values are unreliable. An attacker looking at a specific record cannot determine the true value — it might be the original ± noise.
- Aggregate statistics are accurate. The noise is symmetric (mean zero), so sums, averages, and distributions over large groups converge to their true values as group size increases.
The calibration depends on the sensitivity of the field — how much a single record can affect the aggregate output. A call duration that ranges from 0 to 3,600 seconds has higher sensitivity than a binary field (answered/unanswered), and requires proportionally more noise to achieve the same privacy guarantee.
Spatial perturbation for location data
For location fields — GPS coordinates in telco, delivery addresses in e-commerce, smart meter locations in energy — spatial perturbation randomly shifts each coordinate within a defined radius. A subscriber's location might be shifted by up to 500 meters in a random direction.
This protects exact positions while preserving approximate geographic patterns. A crowd density analysis at the district level remains accurate; a home address cannot be inferred. However, the perturbation radius must be large enough — a subscriber consistently appearing within 100 meters of the same point at night is still pinpointed. For location data, differential privacy applied to aggregate queries is often more effective than perturbing individual coordinates.
Connection to differential privacy
Perturbation is the mechanism; differential privacy is the guarantee. Standalone perturbation adds noise without answering how much is enough, how privacy degrades across multiple queries over the same data, or what an attacker can actually infer. Differential privacy provides all three — Module 04 covers the formal framework.
Standalone perturbation (without a differential privacy framework) is suitable for internal use cases where a rough privacy improvement is better than none — e.g., adding noise to development/staging datasets. For any use case involving external data sharing, regulatory compliance claims, or formal privacy guarantees, use perturbation within a differential privacy framework so you can quantify and defend the privacy level.
Synthetic data generation
Synthetic data generation creates entirely artificial records that preserve the statistical properties — distributions, correlations, relationships — of the original dataset without containing any real individual's data. When done correctly, no record in the synthetic output corresponds to any real person.
Use cases: ML model training data, dev/test environments, data sharing with partners, and regulatory sandbox experimentation. The main open-source tool for non-DP synthesis is SDV (Synthetic Data Vault) — it supports tabular, relational (multi-table), and time-series synthetic data. For DP-guaranteed synthesis, DPSynth (google/dpsynth) generates synthetic data with formal differential privacy using marginal-based mechanisms (AIM, MST, SWIFT), with both in-memory and Apache Beam pipeline modes for Dataflow-scale processing. More advanced approaches (GANs, LLM-based generators) exist but are harder to operationalize.
The key risk: model memorization
If the generative model memorizes training records rather than learning the underlying distribution, the synthetic output may contain near-exact copies of real individuals. This is most likely for outlier records with unusual attribute combinations (e.g., a rare cell tower pattern combined with high call volume), and for small training datasets.
Membership inference testing validates that this hasn't happened — given a synthetic dataset and a candidate record, can an attacker determine whether that candidate was in the training data? SDV includes privacy evaluation metrics for disclosure risk and attribute inference, but a full membership inference attack requires a separate train/holdout attack model (see Module 12 for the methodology). A well-tested synthetic dataset should show attacker accuracy no better than random guessing.
For externally shared synthetic data, training the model with differential privacy (DP-SGD) provides a formal guarantee that no individual record influenced the output beyond a bounded amount — at the cost of reduced output quality.
Choosing between techniques
No single technique covers all scenarios. The choice depends on five factors: whether re-identification is needed, format constraints, join requirements, the level of analytical utility required, and who will access the output.
Decision factors
Reversibility. Does anyone need to get back to the original value? If yes — for operational re-identification, customer support, Right to Erasure compliance — the technique must be reversible (FPE, tokenization, deterministic encryption). If no, irreversible transformations (hashing, generalization, DP, synthetic data) provide stronger privacy guarantees by eliminating the re-identification surface entirely.
Format constraints. Do downstream systems require the output to look like the original? Legacy billing systems, mediation layers, and third-party integrations with format validation make FPE the pragmatic choice. Greenfield systems without format constraints can use any technique.
Join capability. Do records need to be linked across tables or datasets? Techniques with deterministic output — HMAC (same key), FPE (same key), vaulted tokenization (same vault) — preserve join capability. Techniques with randomized or destructive output — salted hashing (different salts), perturbation, generalization, synthetic data — break joins.
Utility requirements. Individual-record analysis needs techniques that preserve record-level fidelity (pseudonymization, FPE, k-anonymity with moderate generalization). Aggregate analysis can tolerate more aggressive transformations (differential privacy, heavy generalization). ML training can use synthetic data.
Access scope. Internal teams with access controls → pseudonymization + DDM. External partners → structural anonymization (k-anonymity, DP). Public release → strongest guarantees (DP, synthetic data).
Decision tree
Legend: purple = pseudonymization (reversible, still personal data), green = structural anonymization, blue = aggregate/synthetic (strongest privacy).
Techniques are usually combined, not used alone
Production anonymization systems layer multiple techniques because different field types and use cases require different treatments within the same dataset. A typical layered approach for a telco CDR pipeline:
| Field | Technique | Rationale |
|---|---|---|
| MSISDN, IMSI | HMAC-SHA256 (keyed hash) | Pseudonymize direct identifiers; preserve join capability across tables |
| IMEI | HMAC or suppression | Device identifier; no join requirement in most analytics use cases |
| Cell tower ID | Generalization (tower → cluster) | Reduce location precision while preserving regional patterns |
| Call timestamp | Temporal bucketing (→ hourly) | Reduce temporal precision to prevent trajectory reconstruction |
| Call duration | Range bucketing | Reduce behavioral specificity |
| SMS/call content | Full suppression | No analytical value in most use cases; extreme sensitivity |
| Aggregate outputs | Differential privacy (Laplace noise) | Formal guarantee for any externally shared statistics |
The same layering principle applies across verticals:
- Fintech: Account numbers → tokenization (PCI requirement), transaction amounts → range bucketing, merchant names → MCC generalization, aggregate fraud statistics → DP
- E-commerce: Customer IDs → HMAC, product SKUs → category generalization, delivery addresses → postal code prefix, clickstream analytics → DP aggregation
- Energy: Meter IDs → HMAC, smart meter readings → temporal aggregation (15-min → hourly), household locations → grid square, demand forecasting data → synthetic generation
Module 06 shows how these layers map onto a data zones architecture with different treatments per zone.
Summary comparison
| Technique | Reversible | GDPR status | Utility preserved | Join capability | Best for |
|---|---|---|---|---|---|
| Suppression / redaction | No | Anonymization | Low | N/A | Fields with no analytical value |
| Generalization | No | Anonymization (if k-anonymous) | Medium | Reduced | Quasi-identifiers in tabular data |
| Plain hashing | No (but attackable) | Pseudonymization | Medium | Yes | Avoid — vulnerable to dictionary attacks |
| HMAC-keyed hash | No (without key) | Pseudonymization | High | Yes (same key) | Direct identifier pseudonymization |
| Tokenization (vaulted) | Yes (vault) | Pseudonymization | High | Yes (same vault) | High-security identifier replacement |
| FPE | Yes (key) | Pseudonymization | Very high | Yes (same key) | Legacy system compatibility |
| Static masking | No | Depends on technique | Medium | Preserved (with referential consistency) | Non-production environments |
| Dynamic masking | N/A | Personal data | High | N/A | Role-based access control |
| Perturbation / noise | No | Anonymization (if DP) | Medium | No | Numerical fields, location data |
| Synthetic data | No | Not personal data (if no memorization) | High (statistical) | No | ML training, dev/test, data sharing |
Next: Module 04 covers the formal privacy models — k-anonymity, l-diversity, t-closeness, and differential privacy — that provide mathematical guarantees on top of these techniques.
Created by Claude Opus with ❤️ for the Aliz Data Engineering team.