The Airbyte MS SQL Server destination allows you to sync data to SQL Server databases.
Each stream will be output into its own table in SQL Server. Each table will contain 3 columns:
_airbyte_ab_id: a uuid assigned by Airbyte to each event that is processed. The column type in SQL Server is
_airbyte_emitted_at: a timestamp representing when the event was pulled from the data source. The column type in SQL Server is
_airbyte_data: a JSON blob representing with the event data. The column type in SQL Server is
Microsoft SQL Server specifics or why NVARCHAR type is used here:
NVARCHAR is Unicode - 2 bytes per character, therefore max. of 1 billion characters; will handle East Asian, Arabic, Hebrew, Cyrillic etc. characters just fine.
VARCHAR is non-Unicode - 1 byte per character, max. capacity is 2 billion characters, but limited to the character set you're SQL Server is using, basically - no support for those languages mentioned before
Full Refresh Sync
Incremental - Append Sync
Incremental - Deduped History
As this connector does not support dbt, we don't support this sync mode on this destination.
To use the SQL Server destination, you'll need:
MS SQL Server:
Azure SQL Database,
Azure Synapse Analytics,
Azure SQL Managed Instance,
SQL Server 2019,
SQL Server 2017,
SQL Server 2016,
SQL Server 2014,
SQL Server 2012, or
PDW 2008R2 AU34.
Make sure your SQL Server database can be accessed by Airbyte. If your database is within a VPC, you may need to allow access from the IP you're using to expose Airbyte.
You need a user configured in SQL Server that can create tables and write rows. We highly recommend creating an Airbyte-specific user for this purpose.
You will need to choose an existing database or create a new database that will be used to store synced data from Airbyte.
Airbyte supports a SSL-encrypted connection to the database. If you want to use SSL to securely access your database, ensure that the server is configured to use an SSL certificate.
You should now have all the requirements needed to configure SQL Server as a destination in the UI. You'll need the following information to configure the MSSQL destination:
This database needs to exist within the schema provided.
The SSL configuration supports three modes: Unencrypted, Encrypted (trust server certificate), and Encrypted (verify certificate).
Unencrypted: Do not use SSL encryption on the database connection
Encrypted (trust server certificate): Use SSL encryption without verifying the server's certificate. This is useful for self-signed certificates in testing scenarios, but should not be used in production.
Encrypted (verify certificate): Use the server's SSL certificate, after standard certificate verification.
Host Name In Certificate (optional): When using certificate verification, this property can be set to specify an expected name for added security. If this value is present, and the server's certificate's host name does not match it, certificate verification will fail.
Add batch method to insert records
Partial Success in BufferedStreamConsumer
declare object types correctly in spec
Fix doc/params in specification file
Change dockerfile entrypoint
Fix handle symbols unicode
MS SQL Server Destination Release!