How to store GeoJSON in Apache Iceberg tables
Storing raw GeoJSON payloads in a lakehouse table degrades query performance, breaks vectorized execution, and introduces uncontrolled schema drift. The prod…
Read the guideA production reference, not a marketing site
A production-focused resource for implementing, optimizing, and maintaining spatial data in open table formats (Apache Iceberg, Delta Lake). The transition from monolithic spatial databases to a spatial data lakehouse is not a storage migration — it is a fundamental re-architecture of how geospatial data is serialized, versioned, indexed, and queried at scale.
This site documents the engineering contracts required to make that architecture deliver: deterministic geometry serialization (WKB / GeoParquet), partition strategies that align with real query patterns, predicate pushdown that actually pushes down, and Python orchestration that respects snapshot isolation.
Every guide is written for data engineers, platform architects, gis backend developers, and cloud/infrastructure teams. who own the pipeline end-to-end — from S3 buckets and catalog manifests to PySpark configurations and CI/CD validation gates. No vendor pitches, no toy examples; just patterns and trade-offs that survive contact with petabyte-scale data.
Pick a section below to explore deeper topics, troubleshooting matrices, and ready-to-paste configurations.
Start here
The most-referenced engineering deep-dives — battle-tested patterns, troubleshooting matrices, and copy-ready configurations across every section.
How to Run ST_Intersects in DuckDB on GeoParquet
This guide is a complete, runnable recipe for loading DuckDB's spatial extension, reading a GeoParquet file straight from disk or S3, and executing an ST_Int…
Read the guideValidating GeoParquet Metadata in CI
This guide gives you a complete Python plus GitHub Actions recipe that inspects a GeoParquet file's geo metadata — version, primary_column, CRS presence, and…
Read the guideImplementing H3 Hexagon Partitioning in Delta Lake
High-frequency spatial telemetry, mobility grids, and raster tile streams consistently degrade in Delta Lake deployments when partitioned directly by fine-gr…
Read the guideBuilding a Schema Validation Pipeline for Geospatial Tables
This guide builds a complete Python and CI recipe that validates a geospatial table's schema before any write — checking the geometry column type, required C…
Read the guideCompacting Spatial Iceberg Tables with rewrite_data_files
This guide gives a complete, runnable recipe for compacting a small-file-ridden spatial Iceberg table using the rewrite_data_files procedure with a bounding-…
Read the guideGeoParquet vs WKB Column Storage: Trade-offs
This guide settles a concrete storage decision: whether to persist geometry as a GeoParquet-native column carrying file-level geo metadata and a bbox coverin…
Read the guideIceberg vs Delta Lake for Spatial Data
This guide compares Apache Iceberg and Delta Lake specifically for spatial workloads — how each handles geometry, partitioning, clustering, engine access, an…
Read the guideManaging Spatial Schema Evolution in Open Table Formats
Silent geometry drift during schema evolution remains the primary failure vector in production spatial lakehouses. When engineering teams execute ALTER TABLE…
Read the guideBenchmarking Spatial Query Engines on GeoParquet
This guide gives you a single, self-contained Python harness that runs the identical ST_Intersects workload against DuckDB, Trino, and Apache Sedona over one…
Read the guideDelta Lake Spatial Index vs Native GIS Formats: Engineering Deterministic Pruning
In production spatial lakehouse architectures, the most persistent failure mode is silent spatial index invalidation triggered by background compaction and s…
Read the guideReading shapefiles into PyIceberg DataFrames efficiently
The primary failure mode in spatial lakehouse ingestion pipelines is unbounded memory allocation during legacy vector parsing. Shapefiles (.shp, .shx, .dbf)…
Read the guideChoosing an H3 Resolution for Point Data
This guide gives you a repeatable method — plus a runnable script that samples your dataset and prints a recommendation — for picking the H3 resolution that…
Read the guideHow Predicate Pushdown Reduces GIS Query Latency in Spatial Lakehouse Architectures
In spatial data lakehouse deployments, query latency is predominantly driven by compute-side geometry evaluation. When a query engine receives a geospatial f…
Read the guideAsync Catalog Writes with PyIceberg and asyncio
This recipe writes many spatial partitions into an Apache Iceberg table concurrently from a single Python process, using asyncio to overlap Parquet serializa…
Read the guideImplementing Row-Level Security for Geospatial Datasets: Preventing Spatial Index Bypass in Lakehouse Query Planners
In production spatial lakehouse deployments, deterministic access control is a non-negotiable infrastructure requirement. The primary failure mode occurs whe…
Read the guideBroadcast Spatial Joins with Apache Sedona
This guide gives you a complete PySpark and Apache Sedona recipe that performs a broadcast spatial join — a small reference layer is broadcast to every execu…
Read the guideRow-Level Security in Databricks Unity Catalog for GIS Data
This recipe implements row-level security on a spatial Delta table in Databricks Unity Catalog by attaching a SQL ROW FILTER function that restricts rows to…
Read the guideH3 vs S2 vs Geohash for Lakehouse Partitioning
This guide runs H3, S2, and geohash head-to-head on the same points, measures the partition cardinality each produces at comparable cell sizes, and turns the…
Read the guideDetecting CRS Drift in Ingestion Pipelines
This guide gives you a complete Python recipe that inspects each incoming batch, compares its declared coordinate reference system against what the coordinat…
Read the guideRow-Level Security for Spatial Data in Trino
This guide delivers a runnable recipe that enforces row-level security on a spatial Iceberg table in Trino by combining a system access control provider with…
Read the guideSpatial Joins Across Catalogs with Trino
This guide gives you a single runnable Trino SQL recipe that spatially joins a large fact table in an Iceberg catalog against reference geometries in a separ…
Read the guideMapping UTM Zones to Iceberg Partition Columns: Resolving Spatial Skew and Predicate Pushdown Failures
In production spatial lakehouse architectures, partitioning by Universal Transverse Mercator (UTM) zones appears geographically intuitive but consistently tr…
Read the guideUsing delta-rs to write spatial parquet files
Unbounded memory consumption and transaction contention are the dominant failure modes when ingesting vector geometries into Delta Lake. The root cause is a…
Read the guideMapping GeoPandas DataFrames to Arrow Schemas
This recipe converts a GeoPandas GeoDataFrame into a PyArrow table under an explicit geometry-as-WKB schema that carries CRS metadata and tolerates null geom…
Read the guideOptimizing Spatial Joins with Iceberg Z-Ordering
In production lakehouse architectures, spatial join failures rarely stem from raw compute exhaustion. They originate from cross-partition shuffle skew. When…
Read the guideScheduling VACUUM for Spatial Delta Tables
This guide provides a complete delta-rs recipe that schedules OPTIMIZE followed by VACUUM with a safe retention window on a spatial Delta table, guarding tim…
Read the guideQuerying Iceberg Tables with the DuckDB Spatial Extension
This guide shows how to read an Apache Iceberg table from DuckDB using the iceberg extension, decode its WKB geometry column with the spatial extension, and…
Read the guideExplore the sections
Spatial Lakehouse Fundamentals & Architecture
Decouple storage, catalog, and compute. Master geometry serialization (WKB/GeoParquet), snapshot semantics, and the Iceberg/Delta trade-offs that govern production spatial stacks.
- GeoParquet Encoding Standards for Lakehouse Tables
- Iceberg Spatial Type Support: Production Architecture & Operational Guide
- Delta Lake Geometry Handling: Partitioning, Indexing, and Operational Workflows
- Open Table Format Versioning in Spatial Lakehouse Architectures
- CRS Management Pipelines for Spatial Lakehouses
- Security Boundaries for GIS Data
Spatial Partitioning & Indexing Strategies
Hierarchical grids, Z-ordering, Hilbert curves, predicate pushdown, and raster/vector hybrid layouts engineered for sub-second queries at petabyte scale.
- Predicate Pushdown Optimization
- Selecting a Discrete Global Grid for Lakehouse Partitioning
- Bucket Mapping for Raster Data
- Z-Ordering for Geospatial Queries
- Spatial Partitioning Schemes
Python Ecosystem & Integration Workflows
Arrow schemas, PyIceberg, delta-rs, async catalog orchestration, and CI/CD validation — the Python contract that keeps spatial pipelines reproducible and fast.
- Async Execution Patterns
- Optimizing PyIceberg Spatial Workflows: Partitioning, Maintenance, and CI/CD in the Lakehouse
- Delta-rs Geometry Processing
- Automating Lakehouse Maintenance for Spatial Tables
- DataFrame Mapping Strategies
Spatial Query Engines & Compute Optimization
DuckDB, Trino, and Apache Sedona against lakehouse tables — spatial SQL functions, distributed joins, catalog federation, and the benchmarks that decide which engine owns each workload.
- Benchmarking and Selecting a Spatial Query Engine
- Distributed Spatial Compute with Apache Sedona
- Trino Spatial SQL and Cross-Catalog Federation
- DuckDB Geospatial Analytics on Lakehouse Tables