Jinja2 in dbt: What it is and Why It Exists

dbt usa Jinja2, the Python templating engine, to add programmability to the SQL. Everything that fits between double curly brackets {{ }} is a Jinja expression, everything in between {% %} it is a control statement (if, for, set).

Before running each model in the warehouse, dbt compile the Jinja template in pure SQL. You can see the compiled SQL in the folder target/compiled/ after each dbt run.

Variables: var() and env_var()

dbt provides two functions for accessing variables in SQL code:

var(): Project variables

-- In dbt_project.yml puoi definire variabili globali:
# dbt_project.yml
vars:
  start_date: '2024-01-01'
  lookback_days: 30
  payment_methods: ['credit_card', 'paypal', 'bank_transfer']

-- Usale nei modelli con var():
SELECT *
FROM {{ ref('stg_orders') }}
WHERE created_at >= '{{ var("start_date") }}'::date

-- Puoi sovrascrivere una variabile da CLI:
-- dbt run --vars '{"start_date": "2025-01-01", "lookback_days": 7}'

env_var(): Environment Variables

-- Accedi alle variabili d'ambiente del sistema
SELECT *
FROM {{ source('raw', 'events') }}
WHERE environment = '{{ env_var("DBT_ENVIRONMENT", "development") }}'
-- Il secondo parametro è il valore di default (opzionale)

-- Nei profiles.yml per le credenziali (prattica consigliata):
# profiles.yml
my_profile:
  outputs:
    prod:
      type: snowflake
      account: "{{ env_var('SNOWFLAKE_ACCOUNT') }}"
      password: "{{ env_var('SNOWFLAKE_PASSWORD') }}"

if/else conditions in Templates

Jinja conditional statements allow you to write models that behave differently based on context (environment, variables, type of materialization):

-- models/marts/finance/orders_with_taxes.sql
-- Logica di calcolo tasse diversa per paese

SELECT
    order_id,
    customer_id,
    total_amount,

    {% if var("target_market") == "US" %}
    total_amount * 0.08 AS tax_amount,     -- aliquota USA semplificata
    {% elif var("target_market") == "IT" %}
    total_amount * 0.22 AS tax_amount,     -- IVA italiana
    {% else %}
    total_amount * 0.20 AS tax_amount,     -- aliquota default EU
    {% endif %}

    total_amount + tax_amount AS total_with_tax

FROM {{ ref('stg_orders') }}
WHERE status = 'completed'

is_incremental(): Fundamental Pattern

The built-in macro is_incremental() it is used in incremental models to add the temporal filter only when the model is run in incremental mode (not in full refresh):

-- models/marts/events_daily.sql
{{ config(materialized='incremental', unique_key='event_date') }}

SELECT
    DATE_TRUNC('day', event_timestamp) AS event_date,
    event_type,
    COUNT(*) AS event_count,
    COUNT(DISTINCT user_id) AS unique_users
FROM {{ ref('stg_events') }}

-- Questo blocco viene incluso SOLO nelle esecuzioni incrementali
{% if is_incremental() %}
WHERE event_timestamp > (SELECT MAX(event_date) FROM {{ this }})
{% endif %}

GROUP BY 1, 2

Loop for: Dynamic SQL Generation

Jinja loops are very powerful for generating repetitive SQL without copy-pasting:

-- Genera colonne per i giorni della settimana dinamicamente
SELECT
    customer_id,
    order_date,

    {% for day_num in range(1, 8) %}
    SUM(CASE WHEN DAYOFWEEK(order_date) = {{ day_num }}
             THEN total_amount
             ELSE 0 END) AS revenue_day_{{ day_num }}
    {% if not loop.last %},{% endif %}
    {% endfor %}

FROM {{ ref('stg_orders') }}
GROUP BY 1, 2

-- Pivot di metriche da una lista di variabile
{% set metrics = ['revenue', 'order_count', 'avg_order_value'] %}

SELECT
    month,
    region,
    {% for metric in metrics %}
    SUM(CASE WHEN metric_name = '{{ metric }}' THEN metric_value END) AS {{ metric }}
    {%- if not loop.last %},{% endif %}
    {% endfor %}
FROM {{ ref('metrics_unpivoted') }}
GROUP BY 1, 2

Macros: Reusable SQL Functions

Macros are the code reuse mechanism in dbt: Jinja functions that take parameters and return SQL. They go into the directory macros/.

Simple Macro: Null Value Cleanup

-- macros/utils/safe_divide.sql
-- Divisione sicura che evita division by zero

{% macro safe_divide(numerator, denominator, default_value=0) %}
    CASE
        WHEN {{ denominator }} = 0 OR {{ denominator }} IS NULL
        THEN {{ default_value }}
        ELSE {{ numerator }} / {{ denominator }}
    END
{% endmacro %}

-- Utilizzo nel modello:
SELECT
    customer_id,
    total_revenue,
    order_count,
    {{ safe_divide('total_revenue', 'order_count') }} AS avg_order_value
FROM {{ ref('customer_summary') }}

Advanced Macro: Dynamic UNION ALL Generation

-- macros/union_relations.sql
-- Crea UNION ALL da una lista di ref()

{% macro union_all_tables(relations) %}
    {% for relation in relations %}
        SELECT
            '{{ relation }}' AS source_table,
            *
        FROM {{ ref(relation) }}
        {% if not loop.last %}UNION ALL{% endif %}
    {% endfor %}
{% endmacro %}

-- Utilizzo:
-- {{ union_all_tables(['events_jan', 'events_feb', 'events_mar']) }}

Macro with run_query(): Querying the Warehouse in Macros

-- macros/get_column_values.sql
-- Recupera valori distinti da una colonna per uso in loop

{% macro get_column_values(table, column) %}
    {% set query %}
        SELECT DISTINCT {{ column }}
        FROM {{ ref(table) }}
        ORDER BY 1
    {% endset %}

    {% set results = run_query(query) %}

    {% if execute %}           -- execute è False durante la fase di parsing
        {% set values = results.columns[0].values() %}
        {% do return(values) %}
    {% else %}
        {% do return([]) %}
    {% endif %}
{% endmacro %}

-- Utilizzo per un pivot dinamico:
{% set regions = get_column_values('stg_orders', 'region') %}

SELECT
    order_date,
    {% for region in regions %}
    SUM(CASE WHEN region = '{{ region }}' THEN revenue END) AS revenue_{{ region | lower | replace(' ', '_') }}
    {%- if not loop.last %},{% endif %}
    {% endfor %}
FROM {{ ref('orders_daily') }}
GROUP BY 1

dbt-utils: The Standard Library

dbt-utils it is the most used package in the dbt ecosystem. It provides common macros that any project would probably reinvent from scratch:

# packages.yml
packages:
  - package: dbt-labs/dbt_utils
    version: 1.3.0

# Installa con:
# dbt deps

The Most Used dbt-utils Macros

-- 1. generate_surrogate_key: chiave surrogata da più colonne (hash MD5)
SELECT
    {{ dbt_utils.generate_surrogate_key(['order_id', 'customer_id']) }} AS sk,
    order_id,
    customer_id
FROM {{ ref('stg_orders') }}

-- 2. unpivot: trasforma colonne in righe (simile a UNPIVOT SQL)
{{ dbt_utils.unpivot(
    relation=ref('orders_pivoted'),
    cast_to='float',
    exclude=['order_date', 'customer_id'],
    field_name='metric_name',
    value_name='metric_value'
) }}

-- 3. date_spine: genera una sequenza di date continua (per riempire i gap)
WITH date_spine AS (
    {{ dbt_utils.date_spine(
        datepart="day",
        start_date="cast('2024-01-01' as date)",
        end_date="current_date"
    ) }}
),
orders AS (
    SELECT DATE_TRUNC('day', created_at) AS order_date, SUM(amount) AS revenue
    FROM {{ ref('stg_orders') }}
    GROUP BY 1
)
-- LEFT JOIN per avere 0 anche nei giorni senza ordini
SELECT
    d.date_day,
    COALESCE(o.revenue, 0) AS revenue
FROM date_spine d
LEFT JOIN orders o ON d.date_day = o.order_date

-- 4. pivot: trasforma righe in colonne
{{ dbt_utils.pivot(
    column='status',
    values=['completed', 'pending', 'cancelled'],
    agg='count',
    then_value='order_id'
) }}

Best Practices for Macros

Conclusions and Next Steps

With Jinja and macros, dbt stops being a simple SQL runner and becomes a framework of programmable transformation. Variables make models adaptable to environments, i loops eliminate repetition, macros encapsulate reusable logic.

The next article addresses a crucial topic for performance in production: the materializations. When to use views, tables, incremental models, and snapshots - and how to choose the right strategy for each dataset.