What Are dbt Sources? Declaring, Testing, and Monitoring Raw Data

dbt sources are declarations of the raw tables and views in your data warehouse that your dbt project reads from. Rather than referencing raw tables directly in SQL with hardcoded database and schema names, dbt sources let you declare these raw inputs in YAML configuration files, reference them with a source() function in your models, and test and monitor them as first-class objects in your dbt project.

Why Declare Sources

Without source declarations, your staging models contain raw SQL like:

SELECT * FROM raw.salesforce.accounts

This works, but it has problems. If the raw schema path changes, every model that references it must be updated. There is no mechanism to test that the raw table has the expected columns, that recent data exists, or that referential integrity holds. Lineage in the dbt DAG starts at your staging models — the raw source is invisible in the lineage graph.

With sources declared, the same model uses:

SELECT * FROM {{ source('salesforce', 'accounts') }}

The source() function looks up the table path from the YAML declaration. Changing the database or schema requires one YAML edit, not twenty SQL edits. The raw table appears as a node in the dbt lineage DAG. Source tests run against the raw table before models run. Source freshness checks detect stale data before pipelines produce incorrect results.

Source Declaration Syntax

Sources are declared in schema.yml files (conventionally in the models/ directory, often in a sources/ subdirectory):

sources:

- name: salesforce

database: raw

schema: salesforce

tables:

- name: accounts

description: "Salesforce account records"

columns:

- name: id

tests:

- not_null

- unique

- name: account_name

tests:

- not_null

Each source has a name (the first argument in source()), an optional database override, a schema, and one or more table declarations. Tables can have column-level declarations with tests.

Source Freshness Testing

Source freshness is one of the most valuable features of dbt sources. It detects when data in a raw source table has stopped updating — which could mean a broken EL pipeline, a source system outage, or a delayed feed.

Freshness is configured per table using a loaded_at_field (the timestamp column indicating when records were loaded) and warn_after / error_after thresholds:

freshness:

warn_after: {count: 12, period: hour}

error_after: {count: 24, period: hour}

loaded_at_field: _fivetran_synced

Running dbt source freshness checks the maximum value of loaded_at_field against the current time. If the most recent record is older than 12 hours, dbt warns. Older than 24 hours, dbt errors. Freshness checks run independently from model runs and can be incorporated into CI/CD pipelines to gate model execution: if sources are stale, don't run the downstream models that would produce incorrect results from stale input.

Source Tests

Source tests apply standard dbt tests to raw tables before transformation. This catches data quality issues at the earliest possible point — in the raw layer, before bad data propagates through your entire transformation pipeline.

Common source tests:

- not_null on primary key columns (raw.orders.order_id must never be null)

- unique on primary key columns (no duplicate orders in the raw feed)

- accepted_values for status codes (status must be one of 'pending', 'active', 'closed')

- relationships across sources (every raw.orders.customer_id must exist in raw.customers.customer_id)

Catching a unique violation on raw orders before the staging model runs means you fix the source issue (a broken deduplication step in the EL process) rather than propagating duplicate order keys through your entire dimensional model.

Source vs. Ref

Understanding when to use source() vs. ref() is fundamental to dbt project structure:

- source() references raw tables loaded by EL tools (Fivetran, Airbyte, custom loaders) — tables your dbt project does not manage or create

- ref() references dbt models — tables your dbt project creates and manages

A well-structured dbt project reads raw inputs via source() in staging models only, then every subsequent model uses ref() to reference the output of a staging model or another intermediate/mart model. Raw table names appear in exactly one place in the project — the sources YAML and the staging models that select from them. Every other model references dbt-managed models through ref().

Source Documentation

Source declarations in YAML support description fields at the source, table, and column level. These descriptions populate in dbt docs generate, making the raw layer visible and documented in the dbt documentation site alongside the transformed models.

Good source documentation captures what the EL tool is and where data comes from, what the expected refresh schedule is, any known quirks of the source system (e.g., soft deletes represented as status flag changes rather than row deletions), and which team or system is the authoritative owner of the raw feed.

Our data architecture services includes dbt project structure, source layer design, and data quality testing frameworks across Snowflake, BigQuery, and Redshift. Contact us to discuss your dbt implementation.

A former Microsoft data architect audits your data foundation, identifies your top priorities, and sends you a written plan. Free. No pitch.