Overture Places signals pipeline

A Spark, Iceberg and Airflow pipeline that scores whether tens of millions of businesses are still operating, for the Overture Maps Foundation.

Role: built the pipeline at Sparkgeo
Public write-up: Building a self-healing Places dataset with signals

The problem

Places data goes stale: businesses close, move and rebrand. The Overture Maps Foundation needed a confidence score for whether each place is still operating, built from contributor data feeds.

What I did

• Built a weekly pipeline that ingests private data feeds from contributors including Meta, Uber and Tripadvisor and conflates them with Overture Places. I owned the ingestion for each provider, the data model and the end-to-end data science.
• Designed the spatial-temporal ground-truth strategies for POI data, change detection and events.
• Matched points of interest with H3 spatial indexing, string matching and other matching algorithms.
• Trained an XGBoost model on Spark ML that scores operating-status confidence for tens of millions of places.
• Ran it all on Spark, Apache Iceberg and Airflow.

Result

Monthly signal observations grew from under 1 million to over 50 million between May 2025 and March 2026, covering about 61% of roughly 15.5 million US places. Next, the pipeline extends beyond the US. The signals already feed back into Overture Places, for downstream users and for internal data science modelling.