DataHub APIs

DataHub APIs: Automating Your Metadata Operations

Module 12 of Tier 3 User Training — Duration: 45 minutes
By the end of this module, you will be able to:
- Explain when to use the GraphQL API vs. the REST API
- Write a GraphQL query to search and filter DataHub entities
- Call the REST API to perform bulk operations
- Use browser devtools to discover query shapes from the DataHub UI

The UI Gets You Started. The API Gets Work Done.

The DataHub UI is built for interactive exploration
The API is how you automate at scale
Common use cases:
- Bulk-update descriptions across 500 tables after a schema migration
- Find every asset without an owner and generate a compliance report
- Programmatically certify datasets after a quality check passes
- Integrate DataHub metadata into your internal developer portal or data catalog overlay

DataHub Exposes Two API Surfaces

GraphQL: /api/graphql
REST (OpenAPI v3): /openapi/...
Both require authentication: Authorization: Bearer <your-token>
Personal access tokens are generated in DataHub Settings (Access Tokens)
Swagger UI for REST: /openapi/swagger-ui/index.html
GraphQL Explorer (interactive playground): /api/graphiql

You Do Not Need to Read the Docs to Learn GraphQL

The DataHub UI is almost entirely GraphQL-powered
Open browser devtools, go to the Network tab, filter by "graphql"
Every UI action fires a real GraphQL query you can copy and adapt
The GraphQL Explorer at /api/graphiql lets you run those queries interactively
Auto-complete and schema introspection are built in: press Ctrl+Space anywhere in the editor

Brief: Screenshot of browser devtools open alongside the DataHub UI, Network tab active, filter box showing "graphql", with one GraphQL request row selected and its request payload visible in the preview pane.

This is one of the most practical things I will show you today. Nearly every action you take in the DataHub UI, from searching for a dataset to updating an owner, is powered by a GraphQL query or mutation behind the scenes (login and a handful of tracking calls are the exceptions). Open your browser devtools right now if you want to follow along. Go to the Network tab, filter by "graphql", and then click around in the DataHub UI. You will see real requests with real query bodies and real responses. You can copy any one of those request bodies, paste it into the GraphQL Explorer at /api/graphiql, and run it. That is the fastest way to learn what queries look like for any operation you care about. It is also the safest way to prototype: the Explorer shows you the response shape before you commit anything to a script.

The Core Read Operation: `search`

search takes an entity type, a query string, and optional filters
Filters are key-value pairs: field name and a URN or string value
Useful filter fields: domains, tags, platform, owners, glossaryTerms
Returns a list of matching entities with whichever fields you request
Example: find all datasets in the Finance domain with the Certified tag:

query {
  search(input: {
    type: DATASET,
    query: "*",
    orFilters: [{
      and: [
        { field: "domains", values: ["urn:li:domain:Finance"] },
        { field: "tags", values: ["urn:li:tag:Certified"] }
      ]
    }]
  }) {
    total
    searchResults {
      entity {
        ... on Dataset {
          properties { name }
          platform { name }
          ownership {
            owners {
              owner {
                ... on CorpUser { username }
              }
            }
          }
        }
      }
    }
  }
}

Brief: Screenshot of the GraphQL Explorer at /api/graphiql with this search query typed into the left editor pane and the response JSON visible in the right results pane.

The search query is the workhorse of the GraphQL API. You will use it constantly. The key things to understand are: the type field controls which entity type you are searching across; the query field is a full-text search string (use "*" to match everything and rely entirely on filters); and orFilters lets you narrow by any indexed field (the older filters field still works but is deprecated in favor of orFilters). One thing that trips people up: the value for a domain or tag filter is always a URN, not a display name. Domain and tag URNs are usually system-generated IDs, not the display name you see in the UI, so copy the exact URN from the domain or tag's own page rather than guessing it. We will see this in the lab. The ... on Dataset syntax is a GraphQL inline fragment: it says "if this entity is a Dataset, give me these fields." You will need this pattern whenever you want type-specific fields, including for the owner's username, which sits behind its own CorpUser fragment since an owner can be a user or a group. Note: the filter field "Tag: Certified" in this example refers to a tag that was applied informally to datasets in the training environment. It is different from the formal Verified status badge, which requires a data steward sign-off. A tag named "Certified" is just a label; the Verified badge is a governance decision. Clarify this if participants ask why they are filtering by a tag rather than the status badge.

Mutations: Writing Metadata Back to DataHub

updateDescription: Set or update the description on any entity
addTag / removeTag: Apply or remove a tag from an entity
addOwner / removeOwner: Add or remove an owner from an entity
All mutations take the entity URN as input
Mutations take effect immediately in the production instance you are connected to
The GraphQL Explorer will autocomplete mutation argument shapes

GraphQL Mutations Affect Real Data Immediately

DataHub has no "dry run" mode for GraphQL mutations
A mutation run against your production instance takes effect immediately
There is no undo button
Rule: always develop and test mutations on a dev or staging instance first
Rule: if you are scripting bulk mutations, run against a small sample (5-10 entities) before scaling up
Rule: use search to preview which entities your filters will match before you mutate them

I want to be very direct about this because it is the most common way people get into trouble with the DataHub API. If you write a Python script that loops through 500 datasets and calls updateDescription on each one, and your script has a bug in the description string, you have now written a bad description to 500 datasets in production. There is no rollback button. The safest workflow is: first, run your search query with the same filters you plan to use for the mutation, verify that the entities coming back are exactly the ones you intend to modify, then run the mutation on one entity, verify the result in the UI, and only then scale up. Always use a dev or staging instance for your initial development. If your organization does not have a separate DataHub environment, the local Docker deployment in the training environment is a safe place to test.

The REST API: Built for Bulk

Swagger UI: /openapi/swagger-ui/index.html
Key endpoints for day-to-day automation:
- /openapi/v3/entity/{entityType}/{urn}: GET — fetch one entity
- /openapi/v3/entity/{entityType}: POST — create or update entities; the body is a JSON array, so one call upserts a whole batch
- /openapi/v3/entity/{entityType}/{urn}: DELETE — remove one entity by URN
DELETE removes the entity outright: there is no soft-delete query parameter on this endpoint

Brief: Screenshot of the Swagger UI at /openapi/swagger-ui/index.html with the DELETE /openapi/v3/entity/{entityType}/{urn} endpoint expanded, showing its parameters panel (entityType and urn path parameters).

Match the API Surface to the Job

Use GraphQL when:
- Reading entity metadata (single entity or filtered search)
- Making a targeted mutation to one or a few entities
- Exploring what fields are available on an entity type
- Prototyping: the Explorer gives you instant feedback
Use REST when:
- Upserting more than ~20 entities at once (one POST call, JSON array body)
- Deleting entities in a script: loop one DELETE call per URN

The decision framework is simple: use GraphQL for reading and for targeted writes; use REST when you are writing many entities at once. The crossover point is roughly 20 entities. Below that, a GraphQL mutation loop is fine. Above that, REST wins for upserts because a single POST call carries the whole batch as one JSON array, far fewer HTTP round-trips than a GraphQL mutation per entity. Deletes are the exception: each DELETE call only removes one URN, so bulk deletes still mean one call per entity, but that is easy to loop in a script and still much faster than clicking through the UI. The pro tip about devtools deserves emphasis again: copying queries from the Network tab and adapting them is faster than reading documentation for most tasks. The UI already knows how to talk to the API. Let it teach you.

The `acryl-datahub` Python SDK

Install: pip install acryl-datahub
The SDK wraps the REST API with Python classes
Key classes:
- DataHubRestEmitter: connects to your DataHub instance and emits metadata
- MetadataChangeProposalWrapper: wraps an aspect (description, ownership, tags) in the envelope format DataHub expects
When to use the SDK vs. raw REST:
- SDK: scripting tasks where you want to avoid building JSON payloads by hand

The Python SDK is worth knowing about, especially if your team is already writing Python for data pipelines. It takes care of the request formatting for you: instead of constructing the JSON payload for a MetadataChangeProposal by hand, you instantiate a Python class and the SDK handles serialization. In Custom Integrations we will go deeper on the SDK. For today, the main takeaway is that it exists, it wraps the REST API, and it is faster for scripting tasks than raw REST calls. The two classes you will encounter most are DataHubRestEmitter (which manages the connection and authentication) and MetadataChangeProposalWrapper (which wraps whatever aspect you are writing, such as a description or ownership record). If you prefer to stay in raw HTTP, everything the SDK does can also be done with curl or requests. They are equivalent paths.

Write Queries and Delete Stale Entities

Part 1: GraphQL (20 minutes)
- Open the GraphQL Explorer at /api/graphiql in the training environment
- Write a query that returns all Certified datasets in the Finance domain
- Your result must include: dataset name, platform name, owner username
- Use the example query in your participant guide as a starting point, then modify it
Part 2: REST Delete (10 minutes)
- You have 3 stale entity URNs on your lab card

Brief: Screenshot of the GraphQL Explorer at /api/graphiql with a search query for Certified datasets in the Finance domain typed into the left editor pane and a results list showing dataset name, platform, and owner fields in the response pane.

The lab cards on your tables have the three stale URNs for Part 2 and the training environment URL and token. For Part 1, open the GraphQL Explorer and start with the example query from your participant guide. The example already has the right structure. Your job is to make sure it runs and returns results. If you get zero results, the most common cause is a URN format mismatch on the domain or tag filter: check that the domain URN in your filter matches exactly what is in the training environment. For Part 2, the curl command is in your participant guide verbatim. If you prefer Postman or Python, use those. The important thing is to confirm the entities are gone after the delete by searching for their URNs. I will circulate during the lab. Raise your hand if you hit an error and we will work through it together. Note on "Certified datasets in the Finance domain": in this lab, "Certified" is a tag applied to training datasets — it is not the same as the Verified status badge in a production environment. In production, the equivalent query would filter by status = Verified or by a structured property capturing governance sign-off. Mention this briefly so participants do not carry the habit of tagging assets "Certified" instead of using the Verified workflow.

Check for Understanding

You need to find every dataset in DataHub that has no owner assigned. Which API and which operation do you use?
You have a list of 300 deprecated tables in Snowflake that should be removed from DataHub. Which endpoint and which HTTP method do you use to delete them efficiently?
You want to add a "Reviewed" tag to a single dataset. You are already in the DataHub UI and want to prototype the mutation before writing a script. What is the fastest way to find the correct GraphQL mutation shape?
A colleague says they wrote a mutation script but wants to test it before running it in production. They ask if there is a "preview" mode. What do you tell them?