SQL Server → Firebolt: 1B rows in 12m 19s

So one more benchmark except this time with a really expensive enterprise grade database.

I do want to mention here that Firebolt benchmark is a little different, and slightly slower than what it could possibly be and the reason being that our database server is in GCP while our Firebolt bucket is in AWS. That is definitely a network hop, that will do some latency, as opposed to going from GCP to GCP. This probably explains why Firebolt benchmarks are slightly slower.

We could not compare this benchmark with Airbyte since Airbyte does not allow an open source connector for SQL Server. Entirely understandable. When we tried to configure the connector after spending a long time installing Airbyte we got the following message.

Benchmark Details

As usual we will use our firenibble table with 1 Billion rows. That’s a lot of rows.

Since we could not do an Airbyte benchmark we decided to play a little bit with SQL Server and see how far we can scale our benchmarks as we increased the source SQL Server machine configuration, the target Snowflake configurations and the number of wirekite threads.

Table

As usual we used our standard firenibble table.

The table has 16 columns, one of them is a numeric primary key, few columns are numbers of various shapes and sizes, few strings and few dates. This makes the table pretty simple to be absorbed by most databases.

CREATE TABLE dbo.firenibble (
    f0 bigint NOT NULL,
    f1 bigint DEFAULT NULL,
    f2 bigint DEFAULT NULL,
    f3 int DEFAULT NULL,
    f4 float DEFAULT NULL,
    f5 float DEFAULT NULL,
    f6 float DEFAULT NULL,
    f7 float DEFAULT NULL,
    f8 varchar(128) DEFAULT NULL,
    f9 varchar(128) DEFAULT NULL,
    f10 date DEFAULT NULL,
    f11 date DEFAULT NULL,
    f12 date DEFAULT NULL,
    f13 varchar(128) DEFAULT NULL,
    f14 varchar(128) DEFAULT NULL,
    f15 varchar(128) DEFAULT NULL,
    PRIMARY KEY (f0)
  );

The table also has 1 Billion rows.

=> select count(*) from dbo.firenibble1;
1: : 1000000000
=>

Loading Data in SQL Server

We used the following command to load the 1 Billion rows in the firenibble table. It took sometime but it worked.

BULK INSERT dbo.firenibble FROM '/mnt/spinning/csv/firenibble.csv' WITH (FIELDTERMINATOR = '\t', ROWTERMINATOR = '\n', FIRSTROW = 1, BATCHSIZE=5000);

Wirekite Run 1

We had a standard 128 CPU GCP instance with 512 GB RAM. The host was Linux Ubuntu 20.04 with SQL Server version 2022. The machine lived in us-central1 region. We rebooted instances between runs.

For Firebolt we created a Medium Size Warehouse also in the us-central1 region to minimize the network distance.

=> ALTER ENGINE my_engine SET TYPE = "M" NODES = 5 MIN_CLUSTERS=1 MAX_CLUSTERS = 2 FAMILY=CO;
ok

We ran with 84 wirekite threads.

And it took about 12 minutes and 35 seconds.

cat wirekite.log | grep -E 'START|FINISH|ELAPSED'
START: Sun Jun 15 05:39:39 UTC 2025
FINISH: Sun Jun 15 05:52:14 UTC 2025
TOTAL ELAPSED TIME: 0 hrs 12 min 35 sec

Wirekite Run 2

This time we ran with 96 threads and got our best time.

Moved 1 billion rows from SQL Server to Firebolt in 12 minutes and 19 seconds.

cat wirekite.log | grep -E 'START|FINISH|ELAPSED'
START: Sun Jun 15 05:39:39 UTC 2025
FINISH: Sun Jun 15 05:52:14 UTC 2025
TOTAL ELAPSED TIME: 0 hrs 12 min 35 sec