Stack Overflow
Stack Overflow

Questions tagged [dbt]

Questions relating to dbt (data build tool) set up and usage. dbt is an open-source command line tool that enables data teams to transform data.

dbt Core is an open-source command line tool that enables data teams to transform data following analytics engineering best practices. It is commercially available on the dbt Cloud platform.

Development:

Stable Version (PyPI)

GitHub Development: dbt-labs/dbt-core (issues)

Official Documentation:

dbt User Docs

dbt Reference Docs

dbt CLI Docs

dbt Cloud Docs

Other resources:

dbt Slack Community - A place for community members to engage in conversational dialogue about specific topics (#redshift, #modeling, etc.)

dbt Discourse Pages - A place for community members to share articles and more "official" dbt core announcements for the dbt Core and dbt Cloud platforms.

1208 questions
15
votes
4 answers

Can dbt connect to different databases in the same project?

It seems dbt only works for a single database. If my data is in a different database, will that still work? For example, if my datalake is using delta, but I want to run dbt using Redshift, would dbt still work for this case?
Willie Chen
  • 160
  • 1
  • 1
  • 9
14
votes
1 answer

DBT custom schema using folder structure

is there a way in DBT to create custom schemas for a model in a derived way by looking at the folder structure? For example, say this is my structure: models └-- product1 └-- team1 | └-- model1.sql └-- team2 └-- model2.sql In…
Catazza
  • 255
  • 3
  • 10
14
votes
1 answer

dbt how to create custom table without using CTAS?

I want to create an empty table with specific columns and data types, I don't have any reference table from where I can do SELECT * FROM . The following link has an image which I intend to do Please find the attached image
Shahrukh lodhi
  • 360
  • 2
  • 13
12
votes
5 answers

while running the dbt run command gets error

connection to dbt and snowfalke was successful but when tried to run this command: $ dbt run it gives this error ERROR: Runtime Error Could not find profile named 'learn_dbt' Encountered an error: Runtime Error Could not run dbt" Am I making any…
Ravi
  • 203
  • 1
  • 3
  • 6
11
votes
3 answers

Configure SQLFluff rules

I use SQLFluff to ensure a uniform syntax in the company and reduce error warnings before running the models in dbt. Since our syntax does not completely match the syntax of SQLFluff I would like to make some changes. The Rules References provided…
Albin
  • 822
  • 1
  • 7
  • 25
11
votes
4 answers

How do we define select statement as a variable in dbt?

Hi I am trying to define a select statement in a set variable in dbt, can any one suggest how to set sql query as a variable in dbt and how to access those variables in below CTEs?
Sweety
  • 191
  • 1
  • 2
  • 5
10
votes
1 answer

dbt ref() vs source()

I’m trying to make one model in dbt depend on another (trying to run the second model after the first one is completely finished), but I’m confused, when to use ref() or source()? What is the difference between them?
Sara
  • 111
  • 1
  • 5
10
votes
1 answer

Using multiple columns in a Unique_Key for incremental loading in DBT

For incremental models, the DBT documentation here says: The unique_key should be supplied in your model definition as a string representing a simple column or a list of single quoted column names that can be used together, for example, ['col1',…
David Garrison
  • 2,546
  • 15
  • 25
10
votes
2 answers

Why do I get a 'select active warehouse' error in dbt when trying the table materialization, but not with the view materialization?

I've been working with dbt for a couple of months now, so still fairly new to it. When running a test model, I have no problems when using the view materialization: {{ config(materialized='view') }} select 1 as id Resulting in: 15:30:25 | 1 of 1…
stuartcoggins
  • 103
  • 1
  • 8
9
votes
1 answer

Configuring raw and analytics databases with dbt

I have been reading dbt's How we configure Snowflake guide which explains the rationale behind having a raw database and an analytics database. Raw data is loaded into your warehouse into raw (e.g. by using Fivetran) and analytics is used by dbt to…
Simon
  • 991
  • 8
  • 30
9
votes
1 answer

How to setup dbt UI for data lineage?

I'm new to the dbt and I'm planning to use dbt cli. One question is how to setup the dbt ui and have such a data lineage graph? I didn't find how to do it here with cli https://docs.getdbt.com/tutorial/create-a-project-dbt-cli.
Even
  • 193
  • 1
  • 8
9
votes
1 answer

How can we work with multiple DBT projects

can anyone share how to organise multiple projects in dbt , given the best practices. My present abstract architecture hierarchy is as follows Analytics…
zafar
  • 129
  • 1
  • 4
9
votes
1 answer

How to run multi-tag selector

I am using dbt 0.18.1 and I follow the documentation about tags however I am curious to know how to run multi-tag selector as arguments. According to this: https://github.com/fishtown-analytics/dbt/pull/1014 Select using a mix of tags, fqns, and…
Mincho
  • 214
  • 3
  • 10
9
votes
2 answers

Materialized View vs Table Using dbt

I'm just onboarding dbt and having gone through the tutorial docs I'm wondering if there's a difference between materializing my transformations as views or tables? I'm using Snowflake as the data warehouse. There's some documentation here that…
Kyle Cheung
  • 113
  • 1
  • 1
  • 5
9
votes
3 answers

Write dbt test for positive values

Is there an easy way to write a test for a column being positive in dbt? accepted_values doesn't seem to work for continuous vatiables. I know you can write queries in ./tests but it looks like an overkill for such a simple thing.
David Masip
  • 2,146
  • 1
  • 26
  • 46
1
2 3
…
80 81