Data Masking and Anonymization of Sensitive Data

Data masking and anonymization represent essential privacy protection techniques that transform sensitive data into non-identifiable or obfuscated versions, allowing organizations to use information in development, testing, analytics, and machine learning environments without exposing the personal or confidential data of data subjects. In an increasingly stringent regulatory context, with legislation such as LGPD in Brazil and GDPR in Europe imposing severe fines for the inadequate exposure of Personally Identifiable Information (PII), and considering that developers frequently need to work with copies of production data containing millions of customer records for debugging, performance testing, and feature development, the absence of adequate protections creates monumental risks of data leakage, regulatory non-compliance, and irreparable damage to corporate reputation. The technical challenge lies in applying transformations that preserve the usefulness of the data for legitimate purposes - maintaining statistical distributions, relationships between tables, and format validations - while eliminating the possibility of re-identifying individuals. This article explores static and dynamic masking techniques, irreversible anonymization methods, reversible pseudonymization with access control, tokenization for reference preservation, differential privacy for protection in aggregations, and practical implementations in SQL databases, NoSQL, and modern data warehouses.

Data Masking vs Anonymization vs Pseudonymization

Data Masking Techniques

1. Substitution

      -- Replaces real data with fictitious but realistic values
      Original: João Silva, [email protected], 123.456.789-00
      Masked: Maria Santos, [email protected], 987.654.321-00
      -- SQL example
      UPDATE users_dev
      SET
      email = CONCAT('user', id, '@example.com'),
      cpf = fake_cpf_generator(),
      name = fake_name_generator();
      

2. Shuffling

      -- Redistributes existing values across records
      -- Preserves data distribution but breaks correlation
      User 1: João, [email protected]
      User 2: Maria, [email protected]
      After shuffling:
      User 1: João, [email protected]  -- Email of user 2
      User 2: Maria, [email protected]  -- Email of user 1
      -- Useful for tests, keeping real values but not linkable
      

3. Character Masking

      -- Hides part of the data, keeps partial visibility
      Original: 1234-5678-9012-3456
      Masked: ****-****-****-3456
      Original: [email protected]
      Masked: j***@email.com
      -- SQL
      SELECT
      CONCAT(
      REPEAT('*', LENGTH(name) - 2),
      SUBSTRING(name, -2)
      ) as masked_name,
      CONCAT(
      SUBSTRING(email, 1, 1),
      '***@',
      SUBSTRING_INDEX(email, '@', -1)
      ) as masked_email;
      

4. Nulling Out

      -- Replaces with NULL or a default value
      -- Use: Extremely sensitive data with no usefulness in tests
      UPDATE users_dev
      SET
      social_security_number = NULL,
      credit_card = NULL,
      password_hash = 'REDACTED';
      

5. Deterministic Hashing

      -- Consistent hash maintains relationships
      -- The same input always generates the same output
      -- Useful for JOINs and FKs
      -- PostgreSQL
      UPDATE users_dev
      SET email = encode(
      digest(email || 'salt-secret', 'sha256'),
      'hex'
      ) || '@masked.com';
      -- The same email always becomes the same hash
      -- Preserves referential integrity
      

Tokenization

      -- Token system with a separate vault
      -- Token mapping stored in a secure location
      -- Original data never leaves the vault
      1. Client sends: CPF 123.456.789-00
      2. Tokenization service returns: TOK_8f3d92a1b4c5
      3. Application stores only the token
      4. To detokenize: Token → Vault → Real value
      -- Advantages:
      - Reversible with authorization
      - Real data isolated in a secure vault
      - PCI-DSS compliance (credit cards)
      -- Node.js example
      const token = await tokenizationService.tokenize({
      type: 'cpf',
      value: '123.456.789-00',
      context: 'user-registration'
      });
      // Store: token = "TOK_8f3d92a1b4c5"
      // Detokenize (requires permission)
      const realValue = await tokenizationService.detokenize(token);
      

Pseudonymization

      -- Replaces direct identifiers with pseudonyms
      -- Retains additional data for usefulness
      -- Reversible via a controlled lookup table
      Original:
      User ID: 12345
      Name: João Silva
      Email: [email protected]
      Age: 35
      Pseudonymized:
      Pseudonym: PSEUDO_9f8e7d6c
      Name: [REMOVED]
      Email: [REMOVED]
      Age: 35  -- Retained for analytics
      Cohort: 30-40  -- Generalized
      -- Lookup table (restricted access)
      PSEUDO_9f8e7d6c → User 12345
      

Irreversible Anonymization

K-Anonymity

      -- Ensures each record is indistinguishable from at least k-1 others
      -- Generalization and suppression of quasi-identifiers
      Original:
      | ZIP Code | Age | Gender | Disease |
      |----------|-----|--------|---------|
      | 12345    | 28  | M      | HIV     |
      | 12346    | 29  | M      | HIV     |
      | 54321    | 35  | F      | Cancer  |
      2-anonymous (k=2):
      | ZIP Code | Age   | Gender | Disease |
      |----------|-------|--------|---------|
      | 1234*    | 25-30 | M      | HIV     |
      | 1234*    | 25-30 | M      | HIV     |
      | 5432*    | 30-40 | F      | Cancer  |
      

Differential Privacy

      -- Adds controlled noise to aggregated queries
      -- Impossible to determine whether an individual is in the dataset
      -- Example: Query "how many users have HIV?"
      Real answer: 150
      DP answer: 150 + Laplace_noise(ε) = 152
      -- ε (epsilon): Privacy budget
      -- small ε = more privacy, less accuracy
      -- large ε = less privacy, more accuracy
      -- Python implementation
      import numpy as np
      def laplace_mechanism(true_answer, sensitivity, epsilon):
      noise = np.random.laplace(0, sensitivity/epsilon)
      return true_answer + noise
      # Query: COUNT(users WHERE condition)
      true_count = 150
      dp_count = laplace_mechanism(true_count, sensitivity=1, epsilon=0.1)
      

Practical Implementation

PostgreSQL Data Masking

      -- Extension: anon (PostgreSQL Anonymizer)
      CREATE EXTENSION anon CASCADE;
      SELECT anon.init();
      -- Declare masking rules
      SECURITY LABEL FOR anon ON COLUMN users.email
      IS 'MASKED WITH FUNCTION anon.fake_email()';
      SECURITY LABEL FOR anon ON COLUMN users.name
      IS 'MASKED WITH FUNCTION anon.fake_first_name() || '' '' || anon.fake_last_name()';
      -- Create masked role
      CREATE ROLE masked_user;
      SELECT anon.start_dynamic_masking();
      GRANT SELECT ON users TO masked_user;
      -- When masked_user queries, they see masked data
      -- Privileged roles see real data
      

Application-Level Masking (Node.js)

      const { faker } = require('@faker-js/faker');
      const crypto = require('crypto');
      class DataMasker {
      maskEmail(email) {
      const [local, domain] = email.split('@');
      return \`\${local[0]}***@\$\`;
      }
      maskCPF(cpf) {
      return cpf.replace(/(\d{3})(\d{3})(\d{3})(\d{2})/, '***.$2.$3-**');
      }
      generateFakeEmail() {
      return faker.internet.email();
      }
      generateFakeName() {
      return faker.person.fullName();
      }
      deterministicHash(value, salt) {
      return crypto
      .createHmac('sha256', salt)
      .update(value)
      .digest('hex');
      }
      }
      // Usage
      const masker = new DataMasker();
      const user = {
      email: '[email protected]',
      cpf: '12345678900'
      };
      const masked = {
      email: masker.maskEmail(user.email),  // j***@email.com
      cpf: masker.maskCPF(user.cpf)  // ***.456.789-**
      };
      

Tools and Solutions

Best Practices

• Multi-layer masking: Database + application + visualization
• Preserve usefulness: Maintain formats, statistical distributions
• Referential consistency: Use deterministic hashing for FKs
• Automated pipelines: Automatic masking on dev/staging refresh
• Re-identification testing: Validate that anonymization is irreversible
• PII documentation: Catalog all sensitive fields
• Access controls: Who can see real vs masked data
• Audit logging: Track access to unmasked data