The Fundamental Distinction

It is important to understand one thing right away: dbt Core and dbt Cloud use the same engine. SQL templates, Jinja macros, YAML tests, configuration files - they are identical. dbt Cloud is not a different product: it is a managed service which wraps dbt Core adding infrastructure and productivity tools.

The choice between the two is not technical but organizational: it depends on the size of the team, on the maturity of the data platform, and how much time and infrastructure you have available to manage internally.

dbt Core: The Open-Source Engine

dbt Core is the open-source project maintained by dbt Labs. It's a Python CLI installable with pip, completely free, without limits on models or users.

-- Installazione dbt Core (esempio con adapter Snowflake)
pip install dbt-snowflake==1.8.0

-- Verificare la versione
dbt --version

-- Cosa ottieni con dbt Core:
-- * dbt run, dbt test, dbt docs, dbt source freshness
-- * Tutti i tipi di materializzazione
-- * Macro e package (packages.yml + dbt deps)
-- * Jinja2 templating completo
-- * Schema.yml per documentazione e test
-- * Adapter per 40+ warehouse (Snowflake, BigQuery, Redshift, Databricks, DuckDB...)
-- * Integrazione con qualsiasi orchestratore (Airflow, Prefect, Dagster)

What dbt Core DOES NOT Include

To use dbt Core in production you need to build and maintain yourself:

• Scheduler and orchestration: Airflow, Prefect, Dagster, or cron jobs
• IDE/editor: VS Code + dbt Power User extension, or any editor
• CI/CD: GitHub Actions, GitLab CI, Jenkins - manual configuration
• Hosting of the dbt docs catalog: GitHub Pages, S3 static website, Netlify
• Artifact management (manifest.json): S3, GCS, or other storage for Slim CI
• Execution Tracking: custom logging, Slack/email alerting
• Multi-environment management: profiles.yml with environment variables

dbt Cloud: The Managed Service

dbt Cloud is dbt Labs' SaaS platform. Provides a complete web interface to develop, schedule, monitor and document dbt projects without managing infrastructure. Pricing is for developer seats (2024):

Feature-by-Feature Comparison

1. Development (IDE and Environment)

# dbt Core: sviluppo in locale
# Setup:
# 1. Clona il repository
# 2. Configura profiles.yml con credenziali warehouse
# 3. Apri VS Code (estensione consigliata: dbt Power User)
# 4. dbt run / dbt test dalla CLI

# Vantaggi: workflow git-native, qualsiasi editor, nessuna dipendenza da internet
# Svantaggi: ogni developer configura il proprio ambiente, onboarding più lento

# dbt Cloud IDE: sviluppo nel browser
# Setup:
# 1. Accedi a cloud.getdbt.com
# 2. Configura il progetto (git repo + warehouse connection)
# 3. Ogni developer ha il proprio schema di sviluppo isolato automaticamente
# 4. Commit, preview, run direttamente nel browser

# Vantaggi: zero setup locale, schema isolati per sviluppatore, preview dei risultati inline
# Svantaggi: dipende dalla connessione internet, meno flessibile per workflows complessi

2. Scheduling and Orchestration

# dbt Core: orchestrazione con Airflow (esempio tipico)
# Richiede setup di Airflow + dbt Airflow provider

from airflow.decorators import dag, task
from airflow.operators.bash import BashOperator
from datetime import datetime

@dag(schedule_interval='0 6 * * *', start_date=datetime(2024, 1, 1))
def dbt_daily_pipeline():

    run_models = BashOperator(
        task_id='dbt_run',
        bash_command='cd /opt/dbt && dbt run --target prod --select tag:daily',
        env={'DBT_PROFILES_DIR': '/opt/dbt'}
    )

    run_tests = BashOperator(
        task_id='dbt_test',
        bash_command='cd /opt/dbt && dbt test --target prod --select tag:daily',
        env={'DBT_PROFILES_DIR': '/opt/dbt'}
    )

    run_models >> run_tests

# dbt Cloud: scheduling con job UI
# Configurazione tramite interfaccia web (o API dbt Cloud)

# Job: "Daily Production Run"
# Schedule: "Every day at 6:00 AM UTC"
# Commands:
#   - dbt deps
#   - dbt source freshness
#   - dbt build --target prod --select tag:daily
# Notifications:
#   - Slack: #data-alerts (on failure)
#   - Email: team@company.com (on failure + success)
# Retry: 1 volta dopo 10 minuti in caso di failure

3. CI/CD and Pull Request Testing

This is one of the areas where dbt Cloud Team brings more added value than to self-hosting:

# dbt Core + GitHub Actions: CI manuale da configurare
# .github/workflows/dbt-ci.yml
name: dbt CI

on:
  pull_request:
    branches: [main]

jobs:
  dbt_ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install dbt
        run: pip install dbt-snowflake==1.8.0
      - name: dbt deps
        run: dbt deps
      - name: dbt build (modificati + downstream)
        run: dbt build --select state:modified+ --defer --state ./prod-artifacts
        env:
          SNOWFLAKE_PASSWORD: ${{ secrets.SNOWFLAKE_PASSWORD }}
          DBT_TARGET: ci

# dbt Cloud Team: CI integrato nella UI
# Configurazione: Project Settings > Continuous Integration
# - Trigger: aperta PR su branch main
# - Commands: dbt build --select state:modified+
# - Schema CI: nome schema automatico (dbt_ci_pr123)
# - Commento automatico sulla PR con risultati (successo/failure + righe fallite)
# - Cleanup automatico schema CI dopo merge/close PR

# Vantaggio chiave: nessuna infrastruttura CI da mantenere
# Gli artifact di produzione (manifest.json) sono gestiti automaticamente da dbt Cloud

4. Lineage and Documentation

The Case for Self-Hosting: When dbt Core is Enough

With an initial setup investment, dbt Core on open-source orchestrators it can do almost everything that dbt Cloud Team does. Here is the typical stack self-hosted for intermediate teams:

# Stack dbt Core self-hosted (per team 3-10 persone)
componenti:
  dbt_core: "1.8.x - motore di trasformazione"
  orchestratore: "Airflow 2.x su EC2/GKE oppure Prefect Cloud free tier"
  ide: "VS Code + dbt Power User extension"
  ci_cd: "GitHub Actions (configurazione manuale ~1 giorno)"
  documentazione: "dbt docs generate → GitHub Pages (gratuito)"
  artifacts_storage: "S3 bucket per manifest.json (Slim CI)"
  monitoring: "Airflow logs + Slack webhook custom"

costo_mensile_stimato:
  orchestratore_airflow_ec2: "~30 EUR/mese (t3.medium)"
  s3_artifacts: "~1 EUR/mese"
  totale: "~31 EUR/mese"

tempo_setup_iniziale: "3-5 giorni (DevOps + dbt engineer)"
tempo_manutenzione: "2-4 ore/mese (aggiornamenti, monitoring)"

Choice Guide: Decision Framework

Quick Answer: Self-hosted dbt Core is sufficient for technical teams with infrastructure already in place; dbt Cloud Team is worth the cost of the developer time for setup and maintenance exceeds the cost of the subscription.

dbt Core → dbt Cloud migration

If you start with dbt Core and then migrate to dbt Cloud, the transition is almost painless because the code is identical. The only thing that changes is where it comes from performed and how credentials are managed:

# Prima: profiles.yml locale
my_project:
  target: dev
  outputs:
    dev:
      type: snowflake
      account: "{{ env_var('SNOWFLAKE_ACCOUNT') }}"
      user: "{{ env_var('SNOWFLAKE_USER') }}"
      password: "{{ env_var('SNOWFLAKE_PASSWORD') }}"
      database: analytics
      warehouse: compute_wh
      schema: dbt_dev_{{ env_var('DBT_USER', 'unknown') }}

# Dopo: connessione configurata nella UI dbt Cloud
# I credentials non sono più in profiles.yml
# dbt Cloud gestisce internamente le variabili d'ambiente
# Il file profiles.yml può essere rimosso o mantenuto solo per uso locale

# L'unico cambio nel codice: il profiles.yml non è deployato su dbt Cloud
# (le connessioni sono configurate tramite UI o API)

dbt Semantic Layer: Only available on Cloud Enterprise

An exclusive feature of dbt Cloud that deserves a separate mention is the Semantic Layer (based on MetricFlow). It's dbt Labs' answer to the problem of "multiple metrics": each analyst calculates the revenue in slightly different way, and each dashboard shows different numbers for the same KPI.

# Il Semantic Layer definisce le metriche una volta sola
# e le espone a qualsiasi tool BI tramite API standardizzata

# dbt Core: puoi definire le metriche in YAML (MetricFlow è open-source)
# ma l'integrazione con BI tool richiede dbt Cloud Enterprise

# In dbt Core, MetricFlow può essere usato via CLI:
pip install dbt-metricflow

# Query metriche da CLI (funziona anche senza Cloud):
mf query \
  --metrics revenue \
  --group-by order_date__month

# Integrazione con BI tool (Tableau, Looker, PowerBI):
# SOLO dbt Cloud Enterprise tramite Semantic Layer API
# (JWT-based authentication, JDBC/ADBC connection string)

Features in Preview: dbt Explorer vs Docs

A direct comparison between the dbt Core catalog and the dbt Cloud catalog:

Practical Considerations: An Evolutionary Path

The most honest advice: you don't have to make a definitive choice right away. Many teams start with dbt Core, build experience and understanding of the project, and then evaluate dbt Cloud when they identify concrete problems that Cloud would solve. Migration costs are minimal because the code is identical.

# Segnali che indicano che è il momento di valutare dbt Cloud Team:
segnali_migrazione:
  - "Onboarding di un nuovo developer richiede >2 giorni per setup locale"
  - "Le PR aspettano il CI per >15 minuti (compute CI troppo alto)"
  - "Gli analisti SQL vorrebbero modificare modelli ma temono la CLI"
  - "Airflow/Prefect occupano il 20%+ del tempo di un engineer"
  - "Il catalogo di documentazione è sempre disallineato dalla realtà"
  - "I PR non hanno check automatici e le rotture vanno in produzione"

# Se hai 3+ di questi segnali, il ROI di dbt Cloud Team è quasi certamente positivo.
# Se hai 0-1 segnali, dbt Core self-hosted è probabilmente sufficiente.

Conclusions and Next Steps

There is no universally best choice. dbt Core is excellent for technical teams who want total control; dbt Cloud repays its cost when the time saved on infrastructure and onboarding it surpasses the subscription.

Most importantly: start with dbt Core. Understand what is missing, what is difficult to manage, and then evaluate whether dbt Cloud solves those specific problems in your context.

The next module addresses the topic of CI/CD in depth: how to configure GitHub Actions with Slim CI to test only modified models, saving up to 90% of CI time compared to a full run.