Data Quality

Data Quality in DataHub

This module covers machine-generated quality signals: automated assertions, freshness checks, and volume monitors
By the end of this session, you will be able to read quality signals on any entity page and configure a basic freshness assertion (DataHub Cloud)

If Metadata in Practice was about the human layer of trust -- the descriptions and tags that people write -- this module is about the automated layer. DataHub lets you define Assertions, which are scheduled checks that run against your data and produce pass/fail signals. Think of them as unit tests for your tables. Just as a software engineer runs tests before shipping code, a data team can configure assertions to run before an analyst trusts a dataset for a board report. Native assertion creation, scheduling, and anomaly detection are DataHub Cloud (Observe) capabilities; DataHub Core (self-hosted) displays assertion results reported in from external tools like dbt tests or Great Expectations. We're going to spend this session understanding what assertions tell you, what they don't tell you, and how to create one yourself.

Assertions: Automated Tests for Your Data

An Assertion is a scheduled check that runs against a dataset and records a pass, fail, or error result
Analogy: unit tests for code, but for data
Example: "This table should have had at least one new row added in the last 25 hours" -- true when checked is PASS (green), false is FAIL (red), and a broken check itself is ERROR (yellow)
Assertions run on a schedule (hourly, daily, etc.) and their history is recorded
Creating and scheduling assertions natively in the DataHub UI is a DataHub Cloud (Observe) capability; DataHub Core (self-hosted) ingests and displays assertion results reported by external tools (dbt tests, Great Expectations, Snowflake DMFs) or the SDK

The unit test analogy is the cleanest way to understand assertions. When a software engineer writes a function, they also write a test that checks the function produces the right output. When a data engineer builds a pipeline, they can write an assertion that checks the output meets expected conditions. The critical difference from traditional data testing frameworks is that DataHub surfaces these results on the entity page directly -- anyone who navigates to the dataset can see whether the last assertion run passed or failed without running a SQL query or checking a CI dashboard. The history is also persisted, so you can see whether this is a new failure or whether this table has been failing freshness checks for three weeks. One scoping note: creating, scheduling, and running these checks from inside DataHub -- the workflow we'll practice today -- is a DataHub Cloud (Observe) feature. On DataHub Core (self-hosted), you still get the entity-page display and history for any assertion results reported in from dbt, Great Expectations, or the API -- you just won't create them from this UI.

Freshness Assertions: "Is This Table Up to Date?"

Freshness assertion: checks that a new row was added (or the table was updated) within a specified time window
Example: "At least one row with a created_at timestamp from the last 25 hours must exist"
Catches: stale tables due to failed pipelines, missed DAG runs, upstream data delays
Best for: tables that are updated on a known schedule (daily, hourly)
Anatomy of a freshness assertion (DataHub Cloud): which column is the timestamp (e.g., created_at, updated_at), and what is the maximum acceptable age (e.g., 25 hours for a daily table)

Brief: Screenshot of the DataHub assertion creation form's Freshness tab, showing the timestamp-column dropdown and the max-age input field filled with an example value ("25 hours").

Volume Assertions: "Does This Table Have the Right Amount of Data?"

Volume assertion (DataHub Cloud): checks that the row count of a table (or the number of new rows added in a window) falls within an expected range
Example: "This table should have between 1,000,000 and 10,000,000 rows," or "at least 500 new rows should have been added in the last 24 hours"
Catches: empty tables, unexpected data loss, upstream truncations, sudden spikes
Watch out: don't set volume thresholds too tight -- a range that fails on a 1% change generates constant noise and gets ignored. Start with a wider range (e.g., 50% below average as the floor, not 5%) and tighten after 2-4 weeks of observed variance
Pro tip: Anomaly Detection replaces fixed thresholds with AI-learned normal ranges (DataHub Cloud; formerly called "Smart Assertions")

Brief: Screenshot of the DataHub Volume assertion configuration panel, showing the row-count-range fields (min/max) filled with example bounds.

Column and Custom SQL Assertions: "Is the Data Internally Consistent?"

Column assertions (DataHub Cloud) check constraints on individual columns: e.g., "the amount column must never be negative," "the user_id column must never be NULL," "the status column must be one of: active, inactive, pending"
Custom SQL assertions (DataHub Cloud) run a query that returns a single numeric value -- the assertion passes when that value meets your configured condition (e.g., equals 0, is between two bounds)
Example: SELECT COUNT(*) FROM orders WHERE order_total < 0 -- condition: equals 0
Most flexible type: can encode any business rule as SQL

Brief: Screenshot of the DataHub assertion-type picker dropdown expanded, showing "Column" and "Custom SQL" as selectable assertion types alongside Freshness and Volume.

Column assertions are column-level integrity checks. They're what you configure when you have explicit business rules about what values a column can contain. NULL checks and non-negative checks are the most common. Custom SQL assertions are the escape hatch -- when no standard assertion type captures your business rule, you write the SQL yourself. The convention is to write a query that counts the violating rows (e.g., a COUNT(*) with a WHERE clause for the bad condition) and set the condition to "equals 0". This means you can encode cross-column rules (e.g., "refund amount must not exceed order total"), temporal rules, or any arbitrary business logic. Custom SQL assertions always require an active connection to the underlying warehouse. Both Column and Custom SQL assertions are DataHub Cloud (Observe) capabilities.

Navigating the Quality Tab

How to get there: entity page > click the Quality tab
What you see: a list of all assertions configured for this entity
Current status for each: PASS (green), FAIL (red), ERROR (yellow), NO STATUS (gray)
The timestamp of the last run; click any assertion to expand its history (timeline of past results)
Entity header badge: if any assertion is currently failing, the entity header shows a warning indicator -- visible without opening the Quality tab

Brief: Screenshot of an entity page's Quality tab, showing the assertion list with status badges (PASS green, FAIL red, ERROR yellow, NO STATUS gray) and one row expanded to show its history timeline.

Always Check the Quality Tab Before a Critical Report

When to always check: before a board presentation, before a quarterly financial close, before a one-time analysis with a hard deadline, before sharing a number in an executive meeting
What to look for: any FAIL on a freshness or volume assertion for the datasets you're using

Brief: Screenshot of an entity page's Quality tab opened directly before a report is pulled, showing all assertions in a PASS (green) state, to illustrate the pre-report check habit.

Passing Assertions Are a Floor, Not a Ceiling

What passing assertions mean: the data met its specific defined constraints the last time the check ran -- nothing more
What passing assertions do NOT mean: the underlying business logic is correct, the dbt model has no bugs, the data is appropriate for your specific use case
An unchecked dataset is not the same as a healthy dataset. No assertions means no signals -- not a clean bill of health

This is the most important conceptual point in this entire module. I've seen teams develop false confidence because their Quality tab is all green. Assertions check what they're configured to check -- no more, no less. A freshness assertion tells you data arrived on time. A volume assertion tells you the right number of rows arrived. Neither assertion tells you anything about whether the values in those rows are correct. That kind of validation -- checking that a revenue formula is right, that a join produced the correct output -- requires business logic tests in your transformation layer (dbt tests, Great Expectations, etc.). DataHub assertions are excellent for infrastructure-level checks: is data arriving? Is it the right size? Does it meet basic integrity constraints? But they are a floor for trust, not a ceiling.

Configure a Freshness Assertion (DataHub Cloud)

1. Navigate to entity page > Quality tab
2. Click "+ Add Assertion"
3. Choose "Freshness"
4. Configure: timestamp column (e.g., created_at, updated_at), max age (e.g., "within the last 25 hours"), run schedule (e.g., daily at 8am)
5. Add a description of what the assertion checks and why (e.g., "checks that at least one row was added in the last 25 hours -- this table is updated daily by the billing_sync DAG")

Brief: Screenshot of the "Add Assertion" modal with Freshness selected, showing the timestamp-column dropdown, max-age field, and run-schedule field mid-fill for a training dataset.

This native "+ Add Assertion" workflow is a DataHub Cloud (Observe) capability. If your organization runs DataHub Core (self-hosted) without Cloud, adapt this lab to reviewing assertion results reported in from an external tool (e.g., a dbt test) instead of creating one from the UI. Walk through this step by step on screen. Have participants configure a freshness assertion on their own training dataset simultaneously. The most common question at this point is "what timestamp column should I use?" The answer is: whichever column represents when the row was created or last modified in the source system. If no such column exists, DataHub can check at the table level using the table's last-modified metadata, but column-based checks are more precise. Emphasize step 5: add a description. An assertion called "Freshness Check" with no description is less useful than one that says "Checks that at least one row was added in the last 25 hours -- this table is updated daily by the billing_sync DAG." Future you and your teammates will thank you.

Freshness + Volume Together Cover More Ground Than Either Alone

Freshness alone catches: stale tables (no new rows at all)
Volume alone catches: empty tables, unexpected size changes
What neither catches alone: a table that updates on time but comes back empty (freshness passes, volume fails); a table that has the right row count but hasn't been updated today (volume passes, freshness fails)
Best practice: for any business-critical daily table, configure both a freshness assertion AND a volume assertion

Which Dataset Would You Use?

Part 1 (5 min): you need to produce a quarterly revenue report by end of day. Three datasets are available: Dataset A (freshness PASS 2 hours ago, volume PASS 2 hours ago), Dataset B (freshness FAIL, last ran 26 hours ago against a daily expectation, volume PASS), Dataset C (no assertions configured)
Which dataset is most trustworthy for your use case? Rank them and write one sentence justifying your ranking
Part 2 (5 min): navigate to your training dataset and configure a freshness assertion (DataHub Cloud). Add a description to the assertion

Brief: Screenshot of three dataset entity-page headers side by side, each showing a different Quality-tab summary badge: Dataset A both green, Dataset B freshness red / volume green, Dataset C with no assertion badge at all.

For Part 1, expected ranking: A is most trustworthy (passing assertions, recent runs), B is next (freshness fail is a yellow flag but at least there is coverage), C is least trustworthy (no signals at all). The key discussion point is Dataset C: the absence of assertions is the most dangerous situation because you simply don't know. Dataset B has a freshness failure, which is a concrete problem, but it also means someone cared enough to set up the check -- you can investigate and escalate. Dataset C gives you nothing to act on. During debrief, push participants on whether they would use Dataset B at all. The right answer is: it depends. If it's for a board presentation, no. If the failure turns out to be a one-time pipeline issue that was resolved this morning, and you can verify the data manually, maybe. Judgment matters.

Incidents — When a Quality Failure Becomes an Operational Event

An Incident is a record that ties a data quality failure to a formal operational status
Fields: severity, title, assignee, resolution timestamp
Created manually from the entity page, or triggered automatically by a failing assertion
Visible on the entity header badge -- consumers see it without opening the Quality tab

Assertions vs. Incidents — Know the Difference

Assertion: automated check -- tells you a rule passed or failed on a schedule
Incident: operational record -- communicates "this is a known active problem being worked"
Assertion fails → you know something is wrong. Creating an Incident → your consumers know something is wrong and someone owns it
Resolving an assertion does NOT automatically close the Incident

What is a Data Contract?

A formalized agreement, attached to a dataset, declaring what it promises to deliver
Aggregates into one governance object: schema shape, freshness SLA, and data quality assertions
Think of it as an SLA card for a dataset -- a published commitment, not a legal document
Contract status is visible as one aggregated summary in the Quality tab's Data Contracts section -- no need to check individual assertion results one by one

Brief: Screenshot of a dataset's Quality tab, Data Contracts section, showing the aggregated SLA card: schema shape, freshness SLA, and the linked quality assertions in one summary view.

Contract Status — Passing vs. Failing

All assertions in the contract passing → contract status green
Any included assertion failing → contract status red
Contract status appears as a single pass/fail summary in the Quality tab's Data Contracts section
More actionable for non-technical stakeholders than a list of individual assertion statuses

Brief: Screenshot of the Quality tab's Data Contracts section showing the contract status card in both states cropped side by side: green "Contract passing" and red "Contract failing."

Define Contracts for Datasets with Downstream SLA Commitments

If a dataset has a documented daily refresh SLA: put it in a contract
Makes SLA compliance visible in one place rather than requiring interpretation of individual assertions
When the contract turns red: clear signal, clear owner, clear resolution path
Start with your highest-traffic Gold-tier datasets

Data Health — The Catalog-Wide Quality View (DataHub Cloud)

Observe > Data Health: aggregate quality view across all entities
Shows: how many assets have assertions, how many are failing, which domains have the most issues
Use for: weekly quality reviews, identifying domains needing assertion coverage, leadership reporting
This is the portfolio view of quality -- not a per-dataset view

Brief: Screenshot of the Observe > Data Health page, showing catalog-wide summary metrics: percentage of assets with assertions, failing-assertion count, and a domains-with-most-issues breakdown.

Before we move on.

What does an ERROR status on an assertion mean? Is the data definitely bad?
A dataset has all freshness and volume assertions passing. Can you conclude the data is correct?
Why should you configure both a freshness AND a volume assertion for a business-critical daily table?
A freshness assertion on FACT_REVENUE_DAILY has been failing for 6 hours and the downstream dashboard shows stale data. Should you resolve the assertion, open an Incident, or both? What is the difference between the two actions?
Your team wants to guarantee FACT_REVENUE_DAILY is refreshed daily and always contains at least 1 million rows. What DataHub feature expresses this as a single published commitment visible on the entity page?

Expected answers: (1) ERROR means the assertion itself had a problem running -- a timeout, a SQL error, a connection failure. You cannot conclude the data is good or bad. It means the check didn't execute, which is a different problem from the data failing a check. You should investigate why the assertion is erroring, not assume the data is fine. (2) No -- assertions only verify what they're configured to check. A passing freshness assertion means data arrived on time. A passing volume assertion means the right number of rows arrived. Neither tells you whether the values in those rows are correct. A broken dbt model with a wrong formula will produce tables with green assertions. (3) Freshness catches stale tables; volume catches empty or anomalous-size tables. A table that updates on schedule but comes back empty will pass freshness and fail volume. Together they cover failure modes that neither catches alone. Q4: Do both -- they address different things. Resolving the assertion marks the technical check as no longer failing once the pipeline catches up. Opening an Incident creates a tracked data quality event with impact, root cause, and a resolution timeline visible across the team. An assertion monitors continuously; an Incident documents a specific failure event for people to act on. Q5: A Data Contract. A Data Contract packages multiple assertions (freshness and volume here) into a single published commitment, with an aggregated pass/fail summary in the dataset's Quality tab. It signals to consumers: this dataset is guaranteed to meet these criteria.

Recap + Looking Ahead

Assertions are scheduled automated checks: freshness, volume, column, custom SQL (native creation is DataHub Cloud; DataHub Core displays externally-reported results)
Status states: PASS (green), FAIL (red), ERROR (yellow), NO STATUS (gray)
Passing assertions are a floor for trust, not a ceiling -- they check what they're configured to check
"No assertions" and "all passing" are different situations
Freshness + volume together cover more ground than either alone
Next: Ownership and Access