Governance Automation

What are Metadata Tests? (DataHub Cloud)

  • Automated governance rules with three parts: Selection, Rules, Actions
  • Selection: which assets this applies to (e.g., 'all Snowflake tables in top 25% of query usage')
  • Rules: what standards those assets must meet (e.g., 'must have an owner AND a description')
  • Actions: what happens on pass or fail (e.g., 'add Needs Documentation tag if fail')
  • Managed under Govern > Tests; run on a schedule — set once, monitors continuously
Brief: Screenshot, the Metadata Test creation page under Govern > Tests, showing the three configuration sections (Selection, Rule, Action) with example values filled in for "all Snowflake tables in top 25% of query usage must have an owner AND a description."

Assertions vs. Metadata Tests — Know the Difference

  • Assertions check your data: freshness, volume, field values
  • Metadata Tests check your metadata: description coverage, owner assignment, domain assignment
  • Complementary — you need both
  • Assertions tell you if the data is healthy; Metadata Tests tell you if governance is healthy
Assertions Freshness Volume Field values Metadata Tests Owner Description Domain Complementary — you need both

Example — High-Usage Assets Must Have Owners

  • Select: top 10% most-queried datasets (by usage metrics)
  • Rule: must have at least one Technical Owner assigned
  • Action: add a 'Needs Owner' tag to any asset that fails the rule
  • Result: any popular dataset without an owner gets tagged automatically and shows up in the failing-assets list — no manual audit needed
Brief: Screenshot, a configured Metadata Test example showing Selection = "top 10% most-queried datasets", Rule = "must have at least one Technical Owner", and Action = "add Needs Owner tag."

Start with Three Tests: Owner, Description, Domain Coverage

  • Owner coverage: does every production dataset have at least one Technical Owner?
  • Description coverage: does every production dataset have a non-empty description?
  • Domain assignment: is every production dataset assigned to a domain?
  • Together, these answer: 'Is this asset governable?' — before asking 'Is the data in it good?'
  • Run across all production datasets; use failures to prioritize curation backlog by query volume
Owner coverage Description coverage Domain assignment Is this asset governable?

What are Compliance Forms? (DataHub Cloud)

  • A structured metadata collection campaign — define questions, select assets, assign to owners
  • Owners receive a task in DataHub to fill in the required fields
  • Tracks completion: see who has responded, who is outstanding, what they said
  • Managed under Govern > Compliance Forms
Brief: Screenshot, the Compliance Forms page under Govern > Compliance Forms, showing a form's completion tracker with counts of respondents vs. outstanding owners.

When to Use a Compliance Form

  • Reach for a Compliance Form when a mandate has a deadline and requires per-asset documented sign-off
  • Three signals this is the right tool: a regulatory or policy deadline is driving the work; completion must be tracked per dataset, not platform-wide; a named data steward must sign off on each asset individually
  • Contrast: Metadata Tests monitor continuous quality. Compliance Forms document one-time or periodic review.
What's driving this? Continuous monitoring Metadata Test Deadline + sign-off Compliance Form

Use Compliance Forms for Time-Bound Governance Mandates

  • Use forms for: regulatory compliance deadlines, quarterly reviews, post-migration cleanup campaigns
  • Pair with Metadata Tests: the test identifies non-compliant assets; the form collects the missing data
  • After the campaign closes: re-run the Metadata Test to verify the gap is closed
  • This gives you a closed-loop governance remediation workflow with an audit trail
Metadata Test identifies gap Compliance Form collects data Re-run test to verify gap closed

Automations — Metadata That Moves Automatically

  • DataHub Automations apply metadata rules automatically — some propagate through the lineage graph, others sync metadata to external tools like Snowflake
  • Add a tag or glossary term to a dataset → automation cascades it downstream automatically
  • Every derived table inherits the classification without manual work
  • Common types: Glossary Term Propagation (DataHub Cloud), Column Documentation Propagation, Snowflake Metadata Sync (DataHub Cloud)
  • Configured under Govern > Automations (DataHub Cloud)
Brief: Screenshot, the Govern > Automations page listing the available automation types (Glossary Term Propagation, Column Documentation Propagation, Snowflake Metadata Sync) with one shown enabled.

Automation Requires Careful Scoping

  • A propagation rule starting at a widely-used raw table will cascade to hundreds of downstream assets
  • Test on a small lineage subgraph first — confirm the blast radius before enabling broadly
  • Always audit what 'downstream' means for your starting entity before turning on propagation
  • Rule: start narrow, review results, expand deliberately
Source table ⚠ Blast radius Table Table Table Table Table ...and many more downstream tables

Configure a Metadata Test (DataHub Cloud)

  • Scenario: Your Finance domain has 15 Production-tagged datasets. Spot-checking shows only 4 have an owner set.
  • Task 1: Create a Metadata Test that flags any Production-tagged dataset in the Finance domain missing an owner
  • Task 2: Name and categorize the test (e.g., "Finance — Ownership Coverage")
  • Task 3: Review the failing assets list — identify which 11 datasets would be flagged on the first run
  • Bonus: Add a second test flagging any dataset with more than 500k rows and no Volume assertion defined
Brief: Screenshot, the Metadata Test creation form pre-filled with Selection = "Domain: Finance AND Tag: Production", ready for the participant to add the owner rule and a name/category.

Check for Understanding

  • Q1: A Metadata Test for "all Production datasets must have an owner" passes on Monday. On Tuesday a dataset is added to Production with no owner. What happens, and when?
  • Q2: Your legal team needs documented proof that 50 finance datasets were reviewed for GDPR compliance by Dec 31. Should you use a Metadata Test, a Compliance Form, or an Automation? Why?
  • Q3: An Automation propagates a glossary term from a parent dataset to all child tables. A new child table is ingested with no glossary term. When does term propagation happen, and what overrides the propagated term?
1 Test evaluation timing 2 Test vs. Form choice 3 Automation propagation timing