Skip to main content

Firebolt

Availability
Available CoreAvailable StandardAvailable PlusAvailable ProAvailable Enterprise FlexAvailable Self-Managed EnterpriseAvailable PyAirbyte
Support Level
Marketplace
Connector Version
0.2.40 (last updated 9 months ago)
CDK Version
0.73.0
Definition ID
18081484-02a5-4662-8dba-b270b582f321

This page guides you through the process of setting up the Firebolt destination connector.

Prerequisites​

  1. Firebolt Account: You need an active Firebolt account. Sign up here if you don’t have one.
  2. Firebolt Database and Engine created: Follow the guide to create an engine if you don't have one in your Firebolt workspace.
  3. Service Account credentials: all programmatic access to Firebolt requires a service account to interact with the database. Create a service account as described in this guide and note its ID and secret.

Replication strategies​

This Firebolt destination connector has two replication strategies:

  1. SQL: Replicates data via SQL INSERT queries. This leverages Firebolt SDK to execute queries directly on Firebolt Engines. Not recommended for production workloads as this does not scale well.

  2. S3: Replicates data by first uploading data to an S3 bucket, creating an External Table and writing into a final Fact Table. This is the recommended loading approach. Requires an S3 bucket and credentials in addition to Firebolt credentials.

Connection parameters​

  • Client ID
    • Your service account id.
  • Client Secret
    • Your service account secret.
  • Account
  • Database
    • The database name where you want the data to be stored.
  • Engine
    • Name of the engine that will run the queries.
  • Host (optional)
    • Firebolt backend URL. Can be left blank for most usecases.

Additionally, for S3 strategy:

  • S3 Bucket Name
    • See this to create an S3 bucket.
  • S3 Bucket Region
    • Create the S3 bucket on the same region as the Firebolt database.
  • Access Key Id
  • Secret Access Key
    • Corresponding key to the above key id.

Setup guide​

  1. Sign up to Firebolt following the guide
  2. Follow the getting started tutorial to setup a database.
  3. Create a service account.
  4. Create an engine as described in here
  5. (Optional) Create a staging S3 bucket (for the S3 strategy).
  6. (Optional) Create an IAM with programmatic access to read, write and delete objects from an S3 bucket.

Supported sync modes​

Sync modeSupported?
Full Refresh - OverwriteYes
Full Refresh - AppendYes
Full Refresh - Overwrite + DedupedNo
Incremental Sync - AppendYes
Incremental Sync - Append + DedupedNo

Connector-specific features & highlights​

Output schema​

Each stream will be output into its own raw Fact table in Firebolt. Each table will contain 3 columns:

  • _airbyte_ab_id: a uuid assigned by Airbyte to each event that is processed. The column type in Firebolt is VARCHAR.
  • _airbyte_emitted_at: a timestamp representing when the event was pulled from the data source. The column type in Firebolt is TIMESTAMP.
  • _airbyte_data: a json blob representing the event data. The column type in Firebolt is VARCHAR but can be parsed with JSON functions.

Namespace support​

This destination does not support namespaces.

Reference​

Config fields reference

Field
Type
Property name
string
account
string
client_id
string
client_secret
string
database
string
engine
string
host
object
loading_method

Changelog​

Expand to review
VersionDatePull RequestSubject
0.2.402025-05-2459866Update dependencies
0.2.392025-05-0359316Update dependencies
0.2.382025-04-2658725Update dependencies
0.2.372025-04-1957623Update dependencies
0.2.362025-04-0557115Update dependencies
0.2.352025-03-2956607Update dependencies
0.2.342025-03-2256156Update dependencies
0.2.332025-03-0855391Update dependencies
0.2.322025-03-0154853Update dependencies
0.2.312025-02-2254217Update dependencies
0.2.302025-02-1553940Update dependencies
0.2.292025-02-0853427Update dependencies
0.2.282025-02-0152946Update dependencies
0.2.272025-01-2549292Update dependencies
0.2.262025-01-1751560Fix connection issues
0.2.252024-11-2548672Update dependencies
0.2.242024-10-2947780Update dependencies
0.2.232024-10-2847100Update dependencies
0.2.222024-10-1246841Update dependencies
0.2.212024-10-0546420Update dependencies
0.2.202024-09-2846144Update dependencies
0.2.192024-09-2145744Update dependencies
0.2.182024-09-1445562Update dependencies
0.2.172024-09-0745245Update dependencies
0.2.162024-08-3144991Update dependencies
0.2.152024-08-2444698Update dependencies
0.2.142024-08-2244530Update test dependencies
0.2.132024-08-1744239Update dependencies
0.2.122024-08-1043682Update dependencies
0.2.112024-08-0343143Update dependencies
0.2.102024-07-2742703Update dependencies
0.2.92024-07-2042211Update dependencies
0.2.82024-07-1341789Update dependencies
0.2.72024-07-1041602Update dependencies
0.2.62024-07-0941118Update dependencies
0.2.52024-07-0640854Update dependencies
0.2.42024-06-2740578Replaced deprecated AirbyteLogger with logging.Logger
0.2.32024-06-2540494Update dependencies
0.2.22024-06-2240078Update dependencies
0.2.12024-06-0639157[autopull] Upgrade base image to v1.2.2
0.2.02024-05-0836443Service account authentication support
0.1.12024-03-0535838Un-archive connector
0.1.02022-05-1813118New Destination: Firebolt
Was this page helpful?