Data engineers handle sensitive data, so protecting it is part of the job. Classify and protect PII, apply masking/tokenization, control access, and meet privacy regulations like GDPR in practice.
You cannot protect what you have not identified, so governance starts by classifying data — which columns are PII (emails, names), sensitive PII (SSNs, health, financial), or public. Classification, recorded in the catalog as metadata, then drives every downstream control: what must be encrypted, masked, access-restricted, or deleted on request. This is the practical, engineering-side response to Security and Privacy on the roadmap.
Classify every column before you can govern it:
PUBLIC product_name, category -> no special handling
INTERNAL order counts, aggregates -> access-controlled
PII email, name, address, phone -> mask/restrict, deletable
SENSITIVE PII SSN, health, payment, biometrics -> encrypt + strict access
Store the classification as METADATA in the catalog. It then DRIVES the
controls: what to encrypt, mask, restrict, and delete on request.Once PII is identified, you limit exposure by transforming it. Masking hides part of a value (show last 4 digits); tokenization replaces it with a reversible token backed by a secure vault; obfuscation/pseudonymization replaces identifiers so analytics work without exposing identity. The right technique depends on whether analysts need to reverse the value and whether they need to join on it.
-- MASKING: show only part, for display / low-trust roles
SELECT
concat('****-****-****-', right(card_number, 4)) AS card_masked,
regexp_replace(email, '(^.).*(@.*$)', '\1***\2') AS email_masked
FROM payments;
-- TOKENIZATION: replace with a reversible token (real value in a secure vault)
-- 4111-1111-1111-1111 -> tok_9f2b... (only the vault can detokenize)
-- HASHING/PSEUDONYMIZATION: stable, non-reversible id for JOINs without identity
-- email -> sha256(email + salt) (analysts can group by user, not see who)
-- pick by need: display=mask, reversible=token, joinable+anonymous=hash.Not everyone should see everything. Warehouses provide role-based access control, column-level and row-level security so you grant the minimum needed — analysts see masked PII, a fraud team sees more, most people see neither. Combined with classification, this enforces "who can see which classification" as policy rather than trust. It is the data-layer application of least privilege.
-- role-based, column- and row-level controls (Snowflake-style shown)
GRANT SELECT ON analytics.fct_orders TO ROLE analyst;
-- column masking policy: analysts see masked email, admins see raw
CREATE MASKING POLICY email_mask AS (val string) RETURNS string ->
CASE WHEN current_role() IN ('PII_ADMIN') THEN val
ELSE regexp_replace(val, '(^.).*(@.*)', '\1***\2') END;
ALTER TABLE customers MODIFY COLUMN email SET MASKING POLICY email_mask;
-- row-level: reps see only their region's rows
CREATE ROW ACCESS POLICY region_policy AS (region string) RETURNS boolean ->
region = current_region() OR is_role_in_session('DATA_ADMIN');
-- least privilege, enforced by the warehouse — not by trust.Regulations like GDPR, CCPA, and the EU AI Act translate into concrete engineering requirements: honor deletion requests (right to erasure), limit retention, record consent and purpose, and keep data in permitted regions. These are why lakehouse table formats support row-level DELETE and why lineage and classification matter — you must find and delete a person’s data everywhere it landed. Treat privacy as a design constraint, not paperwork.
Engineering impact — Privacy law becomes concrete pipeline requirements: deletable records, retention limits, consent/purpose tracking, and regional data residency. Design for them up front.What GDPR/CCPA/EU AI Act mean for your PIPELINES:
right to erasure must DELETE a person everywhere -> lineage + classification
tell you every table; table formats (Delta/Iceberg) allow
row-level DELETE on the lake.
data minimization don't collect/keep more than needed -> retention policies
retention limits auto-expire old PII (TTL / scheduled deletes)
consent + purpose track why you hold data; don't use it beyond that
data residency keep data in permitted regions (cloud region choice)
Design these in from the start — retrofitting "delete this user everywhere"
onto a pipeline with no lineage is brutal.