This is the changelog for Airbyte core. For our connector changelog, please visit our Connector Changelog page.
If you're interested in our progress on the Airbyte platform, please read below!
We now handle nested tables with the normalization steps. Check out the video below to see how it works.
We made some progress to address nested tables in our normalization.
Previously, basic normalization would output nested tables as-is and append a number for duplicate tables. For example, Stripe’s nested address fields go from:
Addressaddress_1
To
Charges_source_owner_755_addresscustomers_shipping_c70_address
After the change, the parent tables are combined with the name of the nested table to show where the nested table originated. This is a breaking change for the consumers of nested tables. Consumers will need to update to point at the new tables.
We now handle nested tables with the normalization steps. Check out the video below to see how it works.
Front-end changes:
Display Airbyte's version number
Describe schemas using JsonSchema
Better feedback on buttons
Add connector build status dashboard
Support Schema Changes in Sources
Support Import / Export of Airbyte Data in the Admin section of the UI
Bug fixes:
If Airbyte is closed during a sync the running job is not marked as failed
Airbyte should fail when instance version doesn't match data version
Upgrade Airbyte Version without losing existing configuration / data
Ability to skip onboarding
Miscellaneous bug fixes:
A long discovery request causes a timeout in the UI type/bug
Out of Memory when replicating large table from MySQL
Increase timeout for long running catalog discovery operations from 3 minutes to 30 minutes to avoid prematurely failing long-running operations
Writing boolean columns to Redshift destination now works correctly
Allow skipping the onboarding flow in the UI
Add the ability to reset a connection's schema when the underlying data source schema changes
Fix UI race condition which showed config for the wrong connector when rapidly choosing between different connector
Fix a bug in MSSQL and Redshift source connectors where custom SQL types weren't being handled correctly. Pull request
Support incremental sync for Salesforce, Slack, and Braintree sources
Gracefully handle invalid nuemric values (e.g NaN or Infinity) in MySQL, MSSQL, and Postgtres DB sources
Fix flashing red sources/destinations fields after success submit
Fix a bug which caused getting a connector's specification to hang indefinitely if the connector docker image failed to download
Tempo
Appstore
You can now deploy Airbyte on Kuberbetes **(alpha version)
Support incremental sync for Mixpanel and Hubspot sources
Fixes on connectors:
Fixed a bug in the GitHub connector where the connector didn’t verify the provided API token was granted the correct permissions
Fixed a bug in the Google Sheets connector where rate limits were not always respected
Alpha version of Facebook marketing API v9. This connector is a native Airbyte connector (current is Singer based).
New source: Plaid (contributed by @tgiardina - thanks Thomas!)
New chat app from the web app so you can directly chat with the team for any issues you run into
Debugging has been made easier in the UI, with checks, discover logs, and sync download logs
Support of Kubernetes in local. GKE will come at the next release.
New source: Looker **
Incremental - Append"
We now allow sources to replicate only new or modified data. This enables to avoid re-fetching data that you have already replicated from a source.
The delta from a sync will be appended to the existing data in the data warehouse.
Here are all the details of this feature.
It has been released for 15 connectors, including Postgres, MySQL, Intercom, Zendesk, Stripe, Twilio, Marketo, Shopify, GitHub, and all the destination connectors. We will expand it to all the connectors in the next couple of weeks.
Other features:
Improve interface for writing python sources (should make writing new python sources easier and clearer).
Add support for running Standard Source Tests with files (making them easy to run for any language a source is written in)
Add ability to reset data for a connection.
Bug fixes:
Update version of test containers we use to avoid pull issues while running tests.
Fix issue where jobs were not sorted by created at in connection detail view.
New sources: Intercom, Mixpanel, Jira Cloud, Zoom, Drift, Microsoft Teams
New destination: our own Redshift warehouse connector. You can also use this connector for Panoply.
New sources: 8 additional source connectors including Recurly, Twilio, Freshdesk. Greenhouse, Redshift (source), Braintree, Slack, Zendesk Support
Bug fixes
Support multiple destinations
New source: Sendgrid
Support basic normalization
Bug fixes
New sources: 10 additional source connectors, including Files (CSV, HTML, JSON...), Shopify, MSSQL, Mailchimp
Here is what we are working on right now:
New destination: our own Snowflake warehouse connector
New sources: Facebook Ads, Google Ads.
New sources: Salesforce, GitHub, Google Sheets, Google Analytics, Hubspot, Rest APIs, and MySQL
Integration test suite for sources
Improve build speed
a new Admin section to enable users to add their own connectors, in addition to upgrading the ones they currently use
improve the developer experience (DX) for contributing new connectors with additional documentation and a connector protocol
our own BigQuery warehouse connector
our own Postgres warehouse connector
simplify the process of supporting new Singer taps, ideally make it a 1-day process
This is our very first release after 2 months of work.
New sources: Stripe, Postgres
New destinations: BigQuery, Postgres
Only one destination: we only support one destination in that 1st release, but you will soon be able to add as many as you need.
Logs & monitoring: you can now see your detailed logs
Scheduler: you now have 10 different frequency options for your recurring syncs
Deployment: you can now deploy Airbyte via a simple Docker image, or directly on AWS and GCP
New website: this is the day we launch our website - airbyte.io. Let us know what you think
New documentation: this is the 1st day for our documentation too
New blog: we published a few articles on our startup journey, but also about our vision to making data integrations a commodity.
Stay tuned, we will have new sources and destinations very soon! Don't hesitate to subscribe to our newsletter to receive our product updates and community news.