Although we support any language to build connectors, many of them are developed in
In order to provide the best developer experience, here are some instructions on how to configure your python project.
Before working with connectors written in Python, we recommend running
./gradlew build from the root project directory. This will create a
virtualenv for every connector and helper project and install dependencies locally.
When iterating on a single connector, you will often iterate by running
This command will:
Install a virtual environment at
Install local development dependencies specified in
At Airbyte, we use IntelliJ IDEA for development. Although it is possible to develop connectors with any IDE, we typically recommend IntelliJ IDEA or PyCharm, since we actively work towards compatibility.
When iterating on a single connector, the easiest approach is to open PyCharm with the desired connector directory.
You should then open the
Preferences > Project: <your connector> > Python Interpreter to configure the Project Interpreter using the one that was installed with the requirements of the connector by the
Our typical development flow is to have one Intellij project for
java development with
gradle and a separate Intellij project for python. The following setup steps are written for IntelliJ IDEA but should have similar equivalents for PyCharm:
Install the Pydantic plugin. This will help autocompletion with some of our internal types.
To create the python project, go to
File -> New -> Project...
Select a project name like
airbyte-python and a directory outside of the
airbyte code root.
Usually you will want to create this project in a new window and not replace the existing window.
Project Structure > Modules.
Click the + sign and
Set the content root and module file location to the location of your
airbyte-integrations directory or a specific subdirectory.
Finish adding the module.
You should now have access to code completion and proper syntax highlighting for python projects.
You can use your default python SDK, but if you want your dependency management to match what will be used in the build process, we recommend creating a Python SDK under
Project Structure > SDKs > + > Virtual Environment > Existing Environment and setting the interpreter to the python script specified in a location such as
airbyte-integrations/connectors/your-connector-dir/.venv/bin/python. Once this is done, you can set the module interpreter to this
venv-based interpreter to make sure imports are working as intended.