Capstone: End-to-End Scenario

The Revenue Discrepancy

  • Scenario: the Finance team reports that last quarter's revenue dashboard showed numbers 12% lower than the actuals in the ERP system
  • The data team suspects an issue in the ETL pipeline feeding the reporting layer
  • Your job: use DataHub to investigate. You will not write any code
  • You have access to the catalog, lineage graph, quality signals, and all entity metadata
  • Time: 20 minutes | Work in pairs (or do this end of day)
ERP System actuals ETL Pipeline suspected issue Revenue Reporting Dashboard ⚑ 12% lower than ERP

Capstone Tasks

  • Using DataHub, complete the following (work in pairs):
  • Task 1 (Search): Find all Production datasets in the Finance domain. How many exist? Which has the most downstream dependents?
  • Task 2 (Lineage): Pick the most-used Finance dataset. Trace its lineage upstream to the raw source. How many hops?
  • Task 3 (Quality): Check the Quality tab on that dataset. Are assertions configured? What is the current status of each?
  • Task 4 (Metadata): Check the description, tags, and glossary terms. Is there enough context to use this dataset safely?
  • Task 5 (Ownership): Who owns this dataset? Group or individual? What domain is it assigned to?
Brief: Screenshot of the DataHub search results page with the Domain filter set to Finance and the Environment filter set to PROD, showing the resulting dataset list -- orients participants to where Task 1 begins. (Downstream dependent counts are not shown in this list; participants find them on each dataset's Lineage tab, per Task 2.)

Capstone Debrief and Discussion

  • Which path through the lineage graph led you toward the issue?
  • Were any quality signals visible on the source dataset before you started looking?
  • Who owned the datasets involved? Was that clearly documented in the catalog?
  • If ownership had been missing, how would you find the right person to contact?
  • What single DataHub feature would have surfaced this problem earliest?
Lineage path that led to the issue Quality signals visible beforehand Ownership documented in the catalog How to find an owner if missing Feature that surfaces this earliest Core lesson: governance gaps are usually metadata gaps