WIP MITExpedited Service Group

Climate Pulse

Weather Data Aggregation from ARPA Regional Forecasts, ECMWF, Airports and Public API with Dashboard

Status
WIP
License
MIT
Category
Expedited Service Group
Updated
GitHub is not a UI string to be translated from Italian into English.

Project status

Description

Climate Pulse is an open-source aggregation and visualization pipeline for meteorological data from European public sources, designed for researchers, data journalists, open-data communities, and SMEs needing access to structured historical weather data without paying commercial API fees (Tomorrow.io, OpenWeatherMap premium, Weather.com enterprise). The aggregated sources include: Italian regional ARPA stations (ARPA Puglia, ARPA Lombardia, etc.) via respectful public endpoint scraping, ECMWF Open Data (European forecasting, ERA5 reanalysis dataset available publicly), METAR/TAF airport international datasets via NOAA Aviation Weather Center, and Copernicus Climate Change Service (C3S) datasets. The architecture combines: FastAPI 0.115+ for REST API + WebSocket, TimescaleDB for time-series storage (hypertable for location + variable + timestamp), Angular 21 SSR for historical dashboard (interactive Leaflet map, time-series plots Plotly, multi-station comparison, configurable alerts), Celery worker for periodic ingestion (cron-style), Redis for caching and rate limiting public API. The data model supports standard WMO variables (temperature, humidity, pressure, wind direction/intensity, precipitation, radiation, cloud coverage) with SI unit normalization. The public API exposes REST endpoints for time-series queries (location/variable/period filtering), real-time heatmap geography for variable, CSV/JSON/Parquet export for offline analysis. Licensed under MIT for maximum reusability (open data community). Target deployment: open-data communities, university researchers (climatology, agriculture), environmental data journalists, hobbyists weather, SMEs in agriculture/ renewable energy.

Key Features

  • Multi-source aggregation

    Regional ARPA stations, IT, ECMWF Open Data, METAR/TAF NOAA, Copernicus C3S, plug-in for new data sources.

  • Time-series database for PostgreSQL

    Table for location, variable and timestamp, compression based on chunks, automatically set with configurable retention policy by source.

  • Public API - FastAPI

    API with OpenAPI 3.1, Redis-based rate limiting, real-time streaming via WebSocket, export to CSV, JSON, or Parquet.

  • Dashboard Angular 21 Server-Side Rendering

    Interactive Map with Station Markers, Plotly Time-Series Graphs, Multi-Station Comparison and Alert Thresholds.

  • Standardized WMO Variables

    Temperature, humidity, pressure, wind, precipitation, irradiance, clouds, units SI normalized, quality flag for source.

  • Worker Ingestion for Celery

    Cron-style ingestion ARPA (15min poll), ECMWF (4 cycles/day), METAR (1h poll), retry strategy + Dead Letter Queue.

  • Respectful public audience

    User Agent identified, conservative rate limiting, HTTP caching, fall back to static snapshots if endpoint is down, robot compliance with robots.txt.

  • Adjustable alert settings

    Webhook + email alert for variable thresholds (temperature > 40°C, wind > 100km/h, precipitation > 50mm/h) by location.

Angular + Spring Boot on top of a PostgreSQL database hosted in the cloud via AWS or Google Cloud Platform

  • FastAPI 0.115+
  • Python 3.12
  • TimescaleDB
  • Angular 21
  • Leaflet
  • Plotly
  • Celery
  • Redis
  • ARPA regional APIs
  • ECMWF Open Data
  • NOAA METAR

Use cases

  • Academic researchers in climate and agriculture looking for structured historical weather data without paywalls.
  • Environmental journalists documenting climate trends (heatwaves, droughts, regional floods)
  • Open data community that wants unified multi-source weather API EU (no commercial lock-in)
  • Precision agriculture or renewable energy (solar/ wind) that optimizes weather-driven operations.

Road Map

  1. v0.1.0Scheduled

    Weather Forecast System Core: ARPAPuglia + ECWMF + BaseAPI

    • Puglia ARPA Scrape Adapter
    • ECMWF Open Data Ingestion
    • Time Scale DB Hypertable Setup
    • FastAPI endpoint /stations and /timeseries
  2. v0.5.0Scheduled

    Multiregional IT + Weather Reports from Airports + Dashboards

    • ARPA Lombardy + Lazio + Sicily adapters
    • METAR/TAF Data Ingestion by NOAA
    • Dashboard with Angular Leaflet and Plotly
    • Thresholds for alerts and webhooks settings
  3. v1.0.0Scheduled

    Copernicus C3S + Export + Production Deploy

    • Copernicus C3S Adapter and Reanalysis Dataset
    • Save as CSV/JSON/Parquet file
    • Rate limiting Redis and API key optional
    • Documentation for MkDocs and Community Setup Guide

Who is it for

  • Tech enthusiasts
  • Junior/ Mid-Level Developer

Quick start

Start the project locally with four commands, requires Docker and GitHub Desktop is not required but a git client like SourceTree or Visual Studio Code's built-in one will do just fine though. 

# Quick start (work-in-progress, repo non ancora pubblicato)
# git clone https://github.com/<tba>/climate-pulse.git
# cd climate-pulse
# docker compose up -d  # timescale + redis + climate-pulse-api + worker
# curl http://localhost:8000/v1/stations?region=puglia

Contribute to the project

Climate Pulse is open source. Open an issue, propose a pull request or leave a star on GitHub: every contribution, no matter how small, is welcome.

  • ⭐ Leave a star if it's useful
  • 🐛 Report bugs or suggest features via issue
  • 🔧 Send a pull request with fixes or improvements
  • 📣 Share the project if you find it interesting
Star on GitHubOpen an issuePull request
← Back to list⭐ View on GitHub