Data Products and Applications

What is a Data Product?

  • Packages related datasets, dashboards, and pipelines into a logical unit representing a business capability
  • Lives inside a Domain; has its own owners and documentation
  • The catalog entry for "what the data team offers to consumers" -- not just a raw table
  • Think: an internal API contract for data, not just an asset record
Domain Data Product Table Dashboard Pipeline owners documentation

Data Products vs. Datasets

  • Dataset: a single table or view -- the physical layer
  • Data Product: a bundle (fact table, summary view, dashboard, pipeline) -- the business capability layer
  • Datasets are the unit of technical accountability; Data Products are the unit of business accountability
  • One Data Product can contain many datasets; by default, a dataset can also belong to more than one Data Product at once
Dataset One table technical accountability many datasets Data Product table + dashboard + pipeline business accountability

Creating a Data Product

  • 1. Navigate to the Domain that will contain the product
  • 2. Click the Data Products tab (only appears on Domain pages -- not on global nav)
  • 3. Click "+ New Data Product," enter a name and description, then create it
  • 4. From the new Data Product's page, click "Add Assets" to bundle in datasets, dashboards, and pipelines
  • 5. Add owners, tags, and glossary terms from the Data Product's sidebar
Brief: Screenshot of a Domain page in DataHub, showing the tab bar (e.g. Assets, Documentation, Data Products) with the "Data Products" tab visible, and the "+ New Data Product" button in the panel beneath it.

Name Data Products After Business Capabilities, Not Tables

  • Good: "Revenue Reporting," "Customer 360 View," "Churn Risk Signals"
  • Bad: "fact_revenue_daily," "rpt_revenue_weekly_v2," "PROD.FINANCE.FACT_REV"
  • The name should describe what the consumer gets, not how the engineer built it
  • Name it the way a business analyst would search for it, not the way a SQL query would reference it
Good Revenue Reporting Customer 360 View Churn Risk Signals Bad fact_revenue_daily rpt_revenue_weekly_v2 PROD.FINANCE.FACT_REV

What is an Application? (DataHub Cloud)

  • Groups related data assets around a single business purpose -- not a technical entity for software services
  • Sits between Domains and Data Products: Domain (business area) > Application (one purpose within it) > Data Product (a specific data offering)
  • A DataHub Cloud feature, disabled by default -- an admin turns it on under Settings > Appearance > Show Applications
  • Carries the same governance metadata as other entities: owners, domain, tags, structured properties
  • Most teams don't need this extra layer -- start with Domains and Data Products, and add Applications only if you need more structure between them
Domain Application Data Product Data Product

Applications in Practice (DataHub Cloud)

  • Payments Domain: everything related to processing payments across the business
  • Refund Processing Application: groups the assets used to handle customer refunds -- pipelines, processing datasets, and two Data Products ("Daily Refund Payments," "Daily Refund Failures")
  • Fraud Detection Application: a second Application in the same Domain, grouping the datasets and models used to catch fraudulent transactions
  • Each Application serves one specific purpose; a Domain can contain several
Payments Domain Refund Processing Daily Refund Payments Daily Refund Failures Fraud Detection Real-time Fraud Alerts Weekly Fraud Analytics

Data Product vs. Application -- The Relationship (DataHub Cloud)

  • Both are organizational containers that live inside a Domain -- not a producer/consumer pair joined by lineage
  • Application: the middle tier, one business purpose within a Domain (DataHub Cloud only)
  • Data Product: the smallest tier, a specific data offering -- usually nested inside an Application, but can also sit directly in a Domain
  • An Application can group multiple Data Products; a Data Product can be linked to more than one Application if your organization needs that
Domain Data Product directly in the Domain Application Data Product

Create a Data Product for the Finance Domain

  • Assets to bundle: FACT_REVENUE_DAILY, the Weekly Revenue Summary dashboard, and daily_revenue_pipeline
  • Name: "Revenue Reporting"
  • Write a description answering: what does this represent, who should use it, what SLA does it carry?
  • Assign an owner (use a group, not an individual)
  • Link at least one glossary term
FACT_REVENUE_DAILY Weekly Revenue Summary dashboard daily_revenue_pipeline Revenue Reporting Owner: group 1 glossary term

Check for Understanding

  • Q1: FACT_REVENUE_DAILY is a table in Snowflake. The Finance team packages it with an owner, an SLA, and documentation. Is that package a Data Product or a Dataset, and what is the difference?
  • Q2: Your organization has enabled Applications (DataHub Cloud) and created a "Refund Processing" Application inside the Payments domain with two Data Products nested in it. A teammate on a self-hosted OSS DataHub instance says they can't find the Applications tab anywhere. What's going on?
  • Q3: A critical Data Product has no linked source datasets and no SLA defined. What does a potential consumer learn from that, and what should the data owner do first?
Q1: Data Product vs. Dataset Q2: Applications (DataHub Cloud) Q3: Data Product with no assets or SLA