Technical systems often group their underlying data into namespaces with each namespace's data isolated from another namespace. This isolation allows for better organisation and flexibility, leading to better usability.
An example of a namespace is the RDMS's
schema concept. Some common use cases for schemas are enforcing permissions, segregating test and production data and general data organisation.
The Airbyte Protocol supports namespaces and allows Sources to define namespaces, and Destinations to write to various namespaces.
If a sync is conducted with a Source and Destination that both support namespaces, the source-defined namespace will be the schema in the Destination into which the data is replicated. For such syncs, data is replicated into the Destination in a layout matching the Source.
All of this is automatic, and requires no additional user configuration.
If the Source does not support namespaces, the data will be replicated into the Destination's default namespace.
If the Destination does not support namespaces, the namespace field is ignored.
The following table summarises how this works. We assume the Source contains a data source named
Source supports Namespaces
Destination supports Namespaces
Default Destination Namespace
Final Destination Namespace
Airbyte Stream Name
Example (fully-qualified Source path -> fully-qualified Destination path)
Both Source and Destination connectors need to support namespaces.
Relevant Source and Destination connectors need to be at least version
0.3.0 or later.
0.21.0-alpha or later.
Ability to prefix namespaces.
Ability to configure custom namespaces.
Ability to toggle the namespaces functionality.
Please create a ticket if other namespace features come to mind!
We welcome you to join our Slack if you have any other questions!