Snowflake pushes Postgres CDC into Iceberg with data mirroring

Snowflake published an engineering post on data mirroring, a new public-preview feature for Snowflake Postgres that replicates Postgres tables into Snowflake with low cost, low lag and transactional consistency. Instead of the usual approach, where an external tool pulls changes out of Postgres via logical decoding and knows nothing about the database's actual state, Snowflake flipped the direction: a Postgres extension called snowflake_cdc runs inside Postgres and pushes batches of changes into Apache Iceberg tables in object storage, using background processes called base workers. Because the extension lives inside Postgres, it can coordinate schema changes, complex data manipulation and data definition transactions, and snapshots directly, rather than guessing at them from outside.

Each write to Postgres moves through four stages on a shared timeline: write (the change lands in the WAL), decode (past WAL entries become row-level changes, using Postgres's historic-snapshot facility to read catalog tables as they existed at write time), capture (decoded changes are batched into temporary files), and apply (batches are merged into Snowflake tables). The decoder periodically closes a batch and hands it to the capture process, which appends the finalized files to Iceberg change logs and writes a record to a separate "meta log," also tracking the replicated Postgres log sequence number (LSN). On the Snowflake side, an apply process reads the meta log as a finite state machine and executes its instructions, processing adjacent change batches together per table. If WAL is unexpectedly dropped, Postgres can push a fresh snapshot for Snowflake to consume, a scenario the post says is rare in practice because of failover slots.

The design leans on transactions as the core reliability primitive. Snowflake's first attempt at unifying Postgres and Snowflake was Postgres for your data lake, the managed version of its open-source pg_lake extension, which is now generally available; it lets a single SQL statement delete from a Postgres table and insert into an Iceberg table inside one transaction, after which the data is queryable in Snowflake. Data mirroring extends the same transactional idea to continuous, high-frequency replication: Postgres pushes a batch of data and schema changes into multiple Iceberg change logs within one Postgres-side transaction, and Snowflake merges several batches at a time inside one Snowflake-side transaction, so every Snowflake table advances together, exactly at a Postgres transaction boundary, preserving foreign keys and join correctness.

That transactional design lets Snowflake avoid the conventional CDC pattern of turning every change into an upsert, which the post says causes inconsistent intermediate states, makes inserts expensive to match against a target table in columnar storage, and makes it hard to merge recent changes with existing data efficiently. Because data mirroring instead applies a stream of exactly-once deletions and insertions, insert-heavy workloads, usually a table's largest source of volume, replicate quickly and cheaply since inserts are simply appended rather than upserted.

The last piece is live views, which combine the still-unapplied changes sitting in per-table change logs with the data already in the target tables, pushing filters and projections down into both the Parquet change-log files and the base table scan. That means changes do not need to be applied often to keep lag low: even applying infrequently, the post says live-view lag stays well below a minute, with only a modest increase in query overhead.

Key facts

Data mirroring is a new Snowflake Postgres feature in public preview for replicating Postgres tables into Snowflake with low cost, low lag and transactional consistency.
A Postgres extension called snowflake_cdc runs inside Postgres and pushes change batches into per-table Iceberg change logs and a separate meta log, using background 'base workers', instead of an external tool pulling changes out.
Each write to Postgres passes through four stages on a shared timeline: write, decode, capture and apply.
Postgres for your data lake, the managed version of the open-source pg_lake extension, is now generally available and lets a single SQL transaction move data between Postgres and Iceberg tables.
Live views combine per-table change logs with target tables so query lag stays well below a minute even when changes are applied infrequently.

Why it matters

Standard Postgres change data capture relies on logical decoding pulled by an external tool that has no visibility into Postgres's actual state, so it cannot tell when a schema changed, how a snapshot lines up with subsequent changes, or whether Postgres itself is down versus the network. Snowflake's fix is architectural: put the extension inside Postgres so it always knows the database's state, and push changes out transactionally into Iceberg rather than have an outside system pull them. The post frames this as turning replication from a chaotic process full of failure conditions into what it calls a Swiss clock: no external connectors to fall behind, no snapshots that conflict with changes, no upserts that slow down as tables grow.

Who it affects

Teams running Snowflake Postgres who need their operational data mirrored into Snowflake for analytics, and more broadly anyone maintaining brittle CDC or ETL pipelines between a transactional Postgres database and an analytical warehouse, since the post frames the underlying design problem, transactions being 'out the window' once a pipeline spans systems, as a general one.

How to use it

Data mirroring is in public preview: it is described as always-on, automatic replication that you set up once, after which Postgres tables and their schema changes stay continuously synced to Snowflake with no extra infrastructure to manage. Snowflake also offers Postgres for your data lake, the now generally available managed version of pg_lake, for developers who want direct SQL control over when and how data moves between Postgres and Iceberg tables rather than continuous automatic sync.

How solid is it

This is Snowflake's own engineering account of a feature it built, published on its blog, so it is a first-party description rather than an independently benchmarked report. It goes into real architectural detail, the four-stage write/decode/capture/apply pipeline, the meta log and change log design, live views, and the use of failover slots to make dropped-WAL recovery rare, but it gives essentially one quantitative claim: that live-view lag stays well below a minute even under infrequent applies. No throughput, cost or latency benchmarks are provided, and no individual author is credited; the post speaks throughout in the corporate 'we'.

Risks and caveats

Data mirroring itself is still in public preview, not generally available, unlike pg_lake. The post gives no specific benchmark numbers for throughput, cost or latency beyond the qualitative live-view lag claim, and no general-availability date for data mirroring is stated. Recovery from an unexpected WAL drop depends on failover slots and is described only as 'very rare in practice,' not eliminated, so the failure mode still exists.

“Push-based change data capture avoids a whole class of infrastructure and associated problems, and effectively decouples producer and consumer via object storage.”

— Snowflake engineering blog