We ran a sustained change-data-capture stress test: a full hour of continuous UPDATE traffic through a real Wirekite pipeline — PostgreSQL → Wirekite extractor → Wirekite loader → Firebolt. Wirekite sustained 472,000 changes per second for one hour — 1.80 billion changes captured and landed end to end — with zero validation failures.
This isn’t a short burst. It’s sixty-three minutes of uninterrupted change traffic, measured at every stage of the pipeline, with the loader keeping the target in lockstep the entire time.
The result
| metric | value |
|---|---|
| Total changes captured and landed | 1,799,768,000 (≈1.80 billion) |
| Sustained end-to-end throughput | 472,000 changes/sec |
| Extractor steady-state | 478,000 changes/sec (≈482K typical) |
| Steady-state variance across the hour | ±2% |
| Run duration | 1 h 03 m 31 s |
| Validation | PASS |
The loader never fell behind. It finished landing the final change in Firebolt just 19 seconds after the source stopped producing — no growing backlog, no slow tail at the end.
The workload
A pure UPDATE workload is the hardest sustained shape for a CDC pipeline: every change is a modification to an existing row, the target table stays a fixed size, and the same blocks are rewritten over and over. There’s no “easy” insert-append work to hide behind.
Five identical wide tables, each pre-loaded with 500,000 rows (2.5 million rows total). The row count never changes — every operation is an UPDATE — so each row is rewritten roughly 720 times over the hour.
CREATE TABLE firenibble_N (
id BIGINT PRIMARY KEY,
int_1 INTEGER,
int_2 INTEGER,
int_3 INTEGER,
int_4 INTEGER,
vc_1 VARCHAR(100),
vc_2 VARCHAR(100),
vc_3 VARCHAR(100),
vc_4 VARCHAR(100),
dt_1 DATE,
dt_2 DATE
);The change generator drove the source at a fixed offered rate:
| parameter | value |
|---|---|
| Parallel workers | 64 |
| Offered rate | 500,000 changes/sec |
| Insert / Update / Delete mix | 0% / 100% / 0% |
| Columns touched per update | 50% (5 of the 10 non-key columns) |
| Updated column types | integer |
| Run duration | 3,600 s (one hour) |
At an offered rate of 500,000 changes/sec, the pipeline captured and landed 472,000–478,000 — keeping pace within a few percent of everything the source threw at it, for the full hour.
The hardware
A single AWS EC2 host ran every moving part except Firebolt itself: PostgreSQL, the Wirekite extractor, the Wirekite loader, and the workload generator were all co-resident. The Firebolt engine and its staging lived in the same region.
| Instance | AWS EC2 m7i.16xlarge |
| Processor | Intel Xeon Platinum 8488C (Sapphire Rapids) |
| vCPUs | 64 |
| RAM | 256 GB |
| Region | us-east-1 |
| OS | Ubuntu 24.04 |
| Network | VPC-internal; no public traffic for any pipeline component |
Disk layout
Three independent gp3 EBS volumes, all XFS with noatime:
| volume | size | provisioned IOPS | throughput | role |
|---|---|---|---|---|
| data | 100 GB | 16,000 | 1,000 MB/s | PostgreSQL data directory |
| WAL | 200 GB | 16,000 | 1,000 MB/s | PostgreSQL WAL |
| work | 100 GB | 16,000 | 1,000 MB/s | Wirekite change files + loader staging |
WAL sits on its own larger volume because at ~470,000 changes/sec PostgreSQL produces 1.5–2 GB of WAL per minute; the volume peaked around 104 GB of 200 GB during the hour. The data directory stayed at ~3 GB the whole time — the workload is pure UPDATE on a fixed 2.5-million-row dataset, so the heap never grows.
PostgreSQL configuration
PostgreSQL 17.10, with logical decoding enabled and the cluster sized for 64 vCPUs and 256 GB of RAM:
shared_buffers = 64 GB
effective_cache_size = 192 GB
work_mem = 64 MB
maintenance_work_mem = 4 GB
wal_buffers = 64 MB
wal_level = logical
max_wal_size = 64 GB
min_wal_size = 4 GB
checkpoint_timeout = 30 min
checkpoint_completion_target = 0.9
wal_compression = lz4
random_page_cost = 1.1
effective_io_concurrency = 256
maintenance_io_concurrency = 256
max_worker_processes = 64
max_parallel_workers = 32
max_parallel_workers_per_gather = 8
track_io_timing = onThe Firebolt target
The target was a Firebolt M-size (Medium) engine in us-east-1, co-located with the source host to keep the network distance minimal. The five target tables mirror the source one-to-one (public.firenibble_1 through public.firenibble_5).
Throughput over the hour
Capture rate, measured in 120-second buckets from start through the final drain. The first ~58 minutes are the steady state while the workload is active; minute 60 catches the generator’s final flush; minute 62 is the residual drain after it stopped.
minute changes/sec (band: 470–490K)
0 474 K ████░░░░░░░░░░░░░░░░
2 475 K █████░░░░░░░░░░░░░░░
4 475 K █████░░░░░░░░░░░░░░░
6 475 K █████░░░░░░░░░░░░░░░
8 476 K ██████░░░░░░░░░░░░░░
10 474 K ████░░░░░░░░░░░░░░░░
12 481 K ███████████░░░░░░░░░
14 480 K ██████████░░░░░░░░░░
16 482 K ████████████░░░░░░░░
18 486 K ████████████████░░░░
20 483 K █████████████░░░░░░░
22 480 K ██████████░░░░░░░░░░
24 484 K ██████████████░░░░░░
26 482 K ████████████░░░░░░░░
28 486 K ████████████████░░░░
30 488 K ██████████████████░░
32 478 K ████████░░░░░░░░░░░░
34 483 K █████████████░░░░░░░
36 484 K ██████████████░░░░░░
38 484 K ██████████████░░░░░░
40 482 K ████████████░░░░░░░░
42 482 K ████████████░░░░░░░░
44 482 K ████████████░░░░░░░░
46 482 K ████████████░░░░░░░░
48 483 K █████████████░░░░░░░
50 479 K █████████░░░░░░░░░░░
52 479 K █████████░░░░░░░░░░░
54 480 K ██████████░░░░░░░░░░
56 478 K ████████░░░░░░░░░░░░
58 477 K ███████░░░░░░░░░░░░░
60 513 K ████████████████████ (generator's final flush)
62 72 K ░░░░░░░░░░░░░░░░░░░░ (drain after the generator stopped)The rate is essentially flat for the first ten minutes (~475K), drifts up slightly through the middle of the run as the Intel turbo clocks stabilize and caches warm (peak 488K at minute 30), then settles back to ~480K for the back half.
Steady-state statistics (the active-workload buckets):
| changes/sec | |
|---|---|
| min | 474 K |
| max | 488 K |
| mean | 481 K |
| median (p50) | 482 K |
| p95 | 486 K |
| standard deviation | 3.5 K (0.7% of mean) |
A standard deviation under 1% of the mean is the point: this is a flat line, not a spiky average.
Per-stage rates
Every stage of the pipeline held its rate, and the loader tracked the extractor end-to-end:
| stage | rate |
|---|---|
| generator → PostgreSQL | 500,000 changes/sec (offered) |
| PostgreSQL → Wirekite extractor | 478,000 changes/sec (captured, avg) |
| extractor → loader → Firebolt | tracks the extractor, landing within 19 s of the end |
Resource utilization
The host was barely working:
| resource | peak | utilization |
|---|---|---|
| host CPU (64 vCPU) | ~10 cores busy | ~16% of available compute |
| host RAM (256 GB) | ~80 GB committed | 31% |
| WAL volume (200 GB) | ~104 GB used | 52% |
| data volume (100 GB) | ~3 GB used | pgdata didn’t grow (UPDATE-only) |
| work volume (100 GB) | ~6.6 GB used | loader kept pace; change files didn’t accumulate |
| network egress | low | all traffic intra-region |
The Wirekite extractor itself peaked at roughly 5 of the 64 cores. At 472,000 changes/sec, the machine had most of its compute, memory, and disk untouched.
A more realistic mixed workload
The headline run is 100% UPDATE — the hardest sustained shape, but not the most lifelike one. To characterize a more typical mix, we also ran one-hour workloads of roughly 10% inserts, 82% updates, 8% deletes:
| run | workload | engine | throughput | result |
|---|---|---|---|---|
| A | 10% I / 82% U / 8% D | L single-node | 348,000 changes/sec | PASS |
| B | 10% I / 82% U / 8% D | L single-node | 393,000 changes/sec | PASS |
Both sustained the full hour with zero validation failures. Pure-UPDATE workloads run comfortably on an M engine; for mixed insert/update/delete workloads at higher throughput, a larger single-node engine keeps the pipeline comfortably ahead.
What this means
At 472,000 changes per second sustained, the pipeline has enormous headroom for real workloads:
- A system generating 50 million updates a day — typical for a busy mid-market OLTP database — uses well under 1% of this rate.
- A system at 5 billion updates a day — large fintech or retail scale — sits comfortably inside it.
- Change capture stays near-real-time through bursts: a database averaging 50,000 changes/sec long-run has roughly 10× burst headroom before replication begins to lag.
- Because the loader keeps pace with the source — it finished within 19 seconds of the source stopping — catching up after a brief source outage takes well under a minute.
And this is a conservative number. Everything but Firebolt shared a single 64-vCPU machine — source database, Wirekite, and the workload generator all competing for the same cores. Splitting those across machines, the way a real deployment would, only removes that co-resident competition.