Modifying Domains On Dataset

Why Would You Use Domains?

Domains are curated, top-level folders or categories where related assets can be explicitly grouped. Management of Domains can be centralized, or distributed out to Domain owners Currently, an asset can belong to only one Domain at a time. For more information about domains, refer to About DataHub Domains.

Goal Of This Guide

This guide will show you how to

• create a domain named Marketing
• read domains attached to a dataset fct_users_created.
• add a dataset named fct_users_created to a domain named Marketing.
• remove the domain Marketing from the fct_users_created dataset.

Prerequisites

For this tutorial, you need to deploy DataHub Quickstart and ingest sample data. For detailed steps, please refer to Datahub Quickstart Guide.

Create Domain

GraphQL

mutation createDomain {
  createDomain(input: { name: "Marketing", description: "Entities related to the marketing department" })
}

If you see the following response, the operation was successful:

{
  "data": {
    "createDomain": "<domain_urn>"
  },
  "extensions": {}
}

Curl

curl --location --request POST 'http://localhost:8080/api/graphql' \
--header 'Authorization: Bearer <my-access-token>' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "mutation createDomain { createDomain(input: { name: \"Marketing\", description: \"Entities related to the marketing department.\" }) }", "variables":{}}'

Expected Response:

{ "data": { "createDomain": "<domain_urn>" }, "extensions": {} }

Python

# Inlined from /metadata-ingestion/examples/library/create_domain.py
import logging

from datahub.emitter.mce_builder import make_domain_urn
from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.metadata.schema_classes import ChangeTypeClass, DomainPropertiesClass

log = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)

domain_urn = make_domain_urn("marketing")
domain_properties_aspect = DomainPropertiesClass(
    name="Marketing", description="Entities related to the marketing department"
)

event: MetadataChangeProposalWrapper = MetadataChangeProposalWrapper(
    entityType="domain",
    changeType=ChangeTypeClass.UPSERT,
    entityUrn=domain_urn,
    aspect=domain_properties_aspect,
)

rest_emitter = DatahubRestEmitter(gms_server="http://localhost:8080")
rest_emitter.emit(event)
log.info(f"Created domain {domain_urn}")

Expected Outcomes of Creating Domain

You can now see Marketing domain has been created under Govern > Domains.

Read Domains

GraphQL

query {
  dataset(urn: "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)") {
    domain {
      associatedUrn
      domain {
        urn
        properties {
          name
        }
      }
    }
  }
}

If you see the following response, the operation was successful:

{
  "data": {
    "dataset": {
      "domain": {
        "associatedUrn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)",
        "domain": {
          "urn": "urn:li:domain:71b3bf7b-2e3f-4686-bfe1-93172c8c4e10",
          "properties": {
            "name": "Marketing"
          }
        }
      }
    }
  },
  "extensions": {}
}

Curl

curl --location --request POST 'http://localhost:8080/api/graphql' \
--header 'Authorization: Bearer <my-access-token>' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "{ dataset(urn: \"urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)\") { domain { associatedUrn domain { urn properties { name } } } } }", "variables":{}}'

Expected Response:

{
  "data": {
    "dataset": {
      "domain": {
        "associatedUrn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)",
        "domain": {
          "urn": "urn:li:domain:71b3bf7b-2e3f-4686-bfe1-93172c8c4e10",
          "properties": { "name": "Marketing" }
        }
      }
    }
  },
  "extensions": {}
}

Python

Coming Soon!

Add Domains

GraphQL

mutation setDomain {
    setDomain(domainUrn: "urn:li:domain:marketing", entityUrn: "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)")
}

If you see the following response, the operation was successful:

{
  "data": {
    "setDomain": true
  },
  "extensions": {}
}

Curl

curl --location --request POST 'http://localhost:8080/api/graphql' \
--header 'Authorization: Bearer <my-access-token>' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "mutation setDomain { setDomain(entityUrn: "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)", domainUrn: "urn:li:domain:marketing")) }", "variables":{}}'

Expected Response:

{ "data": { "setDomain": true }, "extensions": {} }

Python

Coming Soon!

With GraphQL

note

Please note that there are two available endpoints (:8000, :9002) to access graphql. For more information about the differences between these endpoints, please refer to DataHub Metadata Service

Expected Outcomes of Adding Domain

You can now see CustomerAccount domain has been added to user_name column.

Remove Domains

GraphQL

mutation unsetDomain {
    unsetDomain(
      entityUrn:"urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)"
    )
}

Expected Response:

{
  "data": {
    "removeDomain": true
  },
  "extensions": {}
}

Curl

curl --location --request POST 'http://localhost:8080/api/graphql' \
--header 'Authorization: Bearer <my-access-token>' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "mutation unsetDomain { unsetDomain(entityUrn: \"urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)\") }", "variables":{}}'

Python

Coming Soon!

Expected Outcomes of Removing Domain

You can now see a domain Marketing has been removed from the fct_users_created dataset.