dbt Pitch

Pitch

Intro

• When is comes to ELT, dbt focuses solely on the “T” transformations of your data within the data warehouse.

• We started with the goal of helping analysts adopt software engineering best practices for data transformation and have…..

• Quickly turned into a single control plane to govern, catalog, orchestrate, and monitor your end-to-end pipelines to promote data trust and collaboration.

Legacy ETL + Stored Procedures

• What we saw from teams building their transformation logic before dbt (with either stored procedures or legacy ETL processes like Informatic/Talend) was that there was more data downtime, increased warehouse costs, and incorrect or unavailable data in production.

• All of this leads to this bloated spaghetti looking pipeline and more stressed and unhappy developers. At the same time, consumers also have difficulty trusting the data.

Transition

So in steps dbt which offers an approach that is self-documenting, testable, and encourages code reusability. 

dbt + Modularity Report

• Modularity in data pipelines is extremely important to us.

• With dbt, each object managed by a data pipeline is defined in a separate model.

• These models are grouped into layers to show the journey from raw to consumption ready data.

• By working this way, teams create reusable, modular components which help avoid duplicating data and confusing development teams.

Outcome

• Ultimately, teams are working towards creating simpler, more transparent data pipelines like the one we see now.

• Another added benefit of working with dbt is tight version control integration.

• The tooling enables teams to test changes to transformation pipelines much faster.

Overall Goal

The overall goal of dbt is to boost speed, scalability and optimization all while reducing the cost of producing insights.