"Good data" is not one thing. Learn the measurable dimensions — completeness, accuracy, consistency, timeliness, validity, uniqueness — and how to turn each into a check you can run.
A pipeline that moves data reliably but delivers wrong numbers is worse than no pipeline — it produces confident, incorrect decisions. Data quality is therefore not a side task but the core of trustworthy data engineering. The first step is to stop treating "quality" as a vibe and break it into specific, measurable dimensions you can assert on.
A pipeline can be 100% reliable and still ship garbage:
ingested on time ✓ but 30% of emails are null -> COMPLETENESS
loaded every row ✓ but amounts are in cents, not $ -> ACCURACY
no errors ✓ but the same order appears twice -> UNIQUENESS
"It ran" != "it's correct." Data quality = making the numbers TRUSTWORTHY.
Break the vague word "quality" into measurable dimensions (next) so you
can test each one.The industry-standard dimensions give you a checklist to reason about any dataset. Each maps to a concrete question and a concrete test: is a required field present (completeness), does it match reality (accuracy), do related values agree (consistency), is it fresh (timeliness), does it fit the rules (validity), and are there no duplicates (uniqueness). Most real data incidents are a failure of one of these.
Dimension Question it answers Example check
COMPLETENESS is required data present? email IS NOT NULL
ACCURACY does it match reality? amount > 0 and in the right unit
CONSISTENCY do related values agree? order.total = sum(order_items)
TIMELINESS is it fresh enough? max(loaded_at) within 2h
VALIDITY does it fit format/rules? status IN ('placed','shipped',...)
UNIQUENESS no unintended duplicates? order_id is unique
Every data test you write asserts ONE of these. Name the dimension, then
write the check.Every dimension becomes a query that counts violations — a result of zero means the check passes. This is the same "select the bad rows" idea as dbt tests, and it works in any warehouse. Writing checks as SQL keeps them close to the data and easy to run in a pipeline or schedule.
-- one check per dimension; each should return 0
-- COMPLETENESS
SELECT count(*) AS null_emails FROM customers WHERE email IS NULL;
-- UNIQUENESS
SELECT count(*) AS dup_orders FROM (
SELECT order_id FROM orders GROUP BY order_id HAVING count(*) > 1
) d;
-- CONSISTENCY (order total matches its line items)
SELECT count(*) AS mismatched FROM orders o
JOIN (SELECT order_id, SUM(qty*unit_price) tot FROM order_items GROUP BY 1) i
ON i.order_id = o.order_id
WHERE o.total <> i.tot;
-- non-zero result = a quality failure to alert on.Real data is rarely perfect, so absolute pass/fail is often too brittle — a single null in a billion rows should not halt a pipeline. Mature quality checks use thresholds ("fail if more than 0.5% of emails are null") and severities (warn vs error), so you catch genuine degradation without crying wolf. This is the same "signal over noise" discipline as software testing.
Absolute checks are brittle at scale:
"0 null emails" -> one null in a billion rows halts prod. Too strict.
Threshold + severity instead:
null_rate(email) > 5% -> ERROR (block the pipeline)
null_rate(email) > 0.5% -> WARN (alert, keep running)
freshness > 6h -> ERROR
Pick thresholds from history (what's normal?), and split WARN vs ERROR so
only real degradation blocks. Noisy checks get ignored — same failure mode
as a flaky test suite.