Skip to main content

Discover data with dbt Explorer

With dbt Explorer, you can view your project's resources (such as models, tests, and metrics) and their lineage to gain a better understanding of its latest production state. Navigate and manage your projects within dbt Cloud to help you and other data developers, analysts, and consumers discover and leverage your dbt resources.

On-demand learning

If you enjoy video courses, check out our dbt Explorer on-demand course and learn how to best explore your dbt project(s)!

Prerequisites

  • You have a dbt Cloud account on the Team or Enterprise plan.
  • You have set up a production or staging deployment environment for each project you want to explore.
  • You have at least one successful job run in the deployment environment. Note that CI jobs do not update dbt Explorer.
  • You are on the dbt Explorer page. To do this, select Explore from the navigation in dbt Cloud.

Overview page beta

Navigate the dbt Explorer overview page to access your project's resources and metadata, available in beta. The page includes the following sections:

  • Search barSearch for resources in your project by keyword. You can also use filters to refine your search results.
  • Sidebar — Use the left sidebar to access model performance, project recommendations in the Project details section. Browse your project's resources, file tree, and database in the lower section of the sidebar.
  • Lineage graph — Explore your project's or account's lineage graph to visualize the relationships between resources.
  • Latest updates — View the latest changes or issues related to your project's resources, including the most recent job runs, changed properties, lineage, and issues.
  • Marts and public models — View the marts and public models in your project.
Access dbt Explorer from dbt Cloud by clicking Explore in the navigation.Access dbt Explorer from dbt Cloud by clicking Explore in the navigation.

Generate metadata

dbt Explorer uses the metadata provided by the Discovery API to display the details about the state of your project. The metadata that's available depends on the deployment environment you've designated as production or staging in your dbt Cloud project.

  • dbt Explorer automatically retrieves the metadata updates after each job run in the production or staging deployment environment so it always has the latest results for your project. This includes deploy and merge jobs.
  • Note that CI jobs do not update dbt Explorer. This is because they don't reflect the production state and don't provide the necessary metadata updates.
  • To view a resource and its metadata, you must define the resource in your project and run a job in the production or staging environment.
  • The resulting metadata depends on the commands executed by the jobs.
To view in ExplorerYou must successfully run
Model lineage, details, or resultsdbt run or dbt build on a given model within a job in the environment
Columns and statistics for models, sources, and snapshotsdbt docs generate within a job in the environment
Test resultsdbt test or dbt build within a job in the environment
Source freshness resultsdbt source freshness within a job in the environment
Snapshot detailsdbt snapshot or dbt build within a job in the environment
Seed detailsdbt seed or dbt build within a job in the environment

Richer and more timely metadata will become available as dbt Cloud evolves.

Explore your project's lineage graph

dbt Explorer provides a visualization of your project’s DAG that you can interact with. To access the project's full lineage graph, select Overview in the left sidebar and click the Explore Lineage button on the main (center) section of the page.

If you don't see the project lineage graph immediately, click Render Lineage. It can take some time for the graph to render depending on the size of your project and your computer’s available memory. The graph of very large projects might not render so you can select a subset of nodes by using selectors, instead.

The nodes in the lineage graph represent the project’s resources and the edges represent the relationships between the nodes. Nodes are color-coded and include iconography according to their resource type.

By default, dbt Explorer shows the project's applied state lineage. That is, it shows models that have been successfully built and are available to query, not just the models defined in the project.

To explore the lineage graphs of tests and macros, view their resource details pages. By default, dbt Explorer excludes these resources from the full lineage graph unless a search query returns them as results.

 How can I interact with the full lineage graph?

Example of full lineage graph

Example of exploring the order_items model in the project's lineage graph:

Example of full lineage graphExample of full lineage graph

Lenses

The Lenses feature is available from your project's lineage graph (lower right corner). Lenses are like map layers for your DAG. Lenses make it easier to understand your project’s contextual metadata at scale, especially to distinguish a particular model or a subset of models.

When you apply a lens, tags become visible on the nodes in the lineage graph, indicating the layer value along with coloration based on that value. If you're significantly zoomed out, only the tags and their colors are visible in the graph.

Lenses are helpful to analyze a subset of the DAG if you're zoomed in, or to find models/issues from a larger vantage point.

 List of available lenses

Example of lenses

Example of applying the Materialization Type lens with the lineage graph zoomed out. In this view, each model name has a color according to the materialization type legend at the bottom, which specifies the materialization type. This color-coding helps to quickly identify the materialization types of different models.

Example of the Materialization type lensExample of the Materialization type lens

Example of applying the Tests Status lens, where each model name displays the tests status according to the legend at the bottom, which specifies the test status.

Example of the Test Status lensExample of the Test Status lens

Keyword search

You can locate resources in your project by performing a keyword search in the search bar. All resource names, column names, resource descriptions, warehouse relations, and code matching your search criteria will be displayed as a list on the main (center) section of the page. When searching for an exact column name, the results show all relational nodes containing that column in their schemas. If there's a match, a notice in the search result indicates the resource contains the specified column. Also, you can apply filters to further refine your search results.

 Search features
 Filters side panel

Example of results from searching on the keyword item and applying the filters models, description, and code:

Example of keyword searchExample of keyword search

Browse with the sidebar

From the sidebar, you can browse your project's resources, its file tree, and the database.

  • Resources tab — All resources in the project organized by type. Select any resource type in the list and all those resources in the project will display as a table in the main section of the page. For a description on the different resource types (like models, metrics, and so on), refer to About dbt projects.
  • File Tree tab — All resources in the project organized by the file in which they are defined. This mirrors the file tree in your dbt project repository.
  • Database tab — All resources in the project organized by the database and schema in which they are built. This mirrors your data platform's structure that represents the applied state of your project.
Example of tabs in sidebarExample of tabs in sidebar

Open in IDE

If you have been assigned a developer license, you can open the resource in the IDE directly from Explorer. For example, the IDE opens all the corresponding files for the model. This includes the model's SQL or Python definition and any YAML files that include an entry for that model. The feature is available from the full lineage graph and the resource's details view.

Here's an example of the Open in IDE icon in the upper right corner of the resource details page. The icon is inactive (grayed out) if you haven't been assigned a developer license.

Example of icon for Open in IDEExample of icon for Open in IDE

View model versions

If models in the project are versioned, you can see which version of the model is being applied — prerelease, latest, and old — in the title of the model’s details page and in the model list from the sidebar.

View resource details

You can view the definition and latest run results of any resource in your project. To find a resource and view its details, you can interact with the lineage graph, use search, or browse the catalog.

The details (metadata) available to you depends on the resource’s type, its definition, and the commands that run within jobs in the production environment.

In the upper right corner of the resource details page, you can:

  • Click the Open in IDE icon to examine the resource using the dbt Cloud IDE.
  • Click the Share icon to copy the page's link to your clipboard.
 What details are available for a model?
 What details are available for an exposure?
 What details are available for a test?
 What details are available for each source table within a source collection?

Example of model details

Example of the details view for the model customers:

Example of resource detailsExample of resource details

Staging environment

dbt Explorer supports views for staging deployment environments, in addition to the production environment. This gives you a unique view into your pre-production data workflows, with the same tools available in production, while providing an extra layer of scrutiny.

You can explore the metadata from your production or staging environment to inform your data development lifecycle. Just set a single environment per dbt Cloud project as “production” or “staging," and ensure the proper metadata has been generated then you’ll be able to view it in Explorer. Refer to Generating metadata for more details.

Explore in a staging environmentExplore in a staging environment
0