The instructions have been tested on Debian GNU/Linux 10 (buster)
Launch a new instance
Configure new instance
For testing out Airbyte, an e2.medium
instance is likely sufficient.
For long-running Airbyte installations, we recommend a n1-standard-2
instance.
Create
This part assumes that you have access to a terminal on your workstation
Set variables in your terminal
# In your workstation terminalPROJECT_ID=PROJECT_ID_WHERE_YOU_CREATED_YOUR_INSTANCEINSTANCE_NAME=airbyte # or anyother name that you've used
Install gcloud
# In your workstation terminalbrew cask install google-cloud-sdkgcloud init # Follow instructions
# FIXME
# Verify you can see your instancegcloud --project $PROJECT_ID compute instances list[...] # You should see the airbyte instance you just created
Connect to your instance
# In your workstation terminalgcloud --project=$PROJECT_ID beta compute ssh $INSTANCE_NAME
Install docker
# In your ssh session on the instance terminalsudo apt-get updatesudo apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-commoncurl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add --sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian buster stable"sudo apt-get updatesudo apt-get install -y docker-ce docker-ce-cli containerd.iosudo usermod -a -G docker $USER
Install docker-compose
# In your ssh session on the instance terminalsudo apt-get -y install wgetsudo wget https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m) -O /usr/local/bin/docker-composesudo chmod +x /usr/local/bin/docker-composedocker-compose --version
Close the ssh connection to ensure the group modification is taken into account
# In your ssh session on the instance terminallogout
Connect to your instance
# In your workstation terminalgcloud --project=$PROJECT_ID beta compute ssh $INSTANCE_NAME
Install Airbyte
# In your ssh session on the instance terminalmkdir airbyte && cd airbytewget https://raw.githubusercontent.com/airbytehq/airbyte/master/{.env,docker-compose.yaml}docker-compose up -d
For security reasons, we strongly recommend to not expose Airbyte publicly. Future versions will add support for SSL & Authentication.
Create ssh tunnel.
# In your workstation terminalgcloud --project=$PROJECT_ID beta compute ssh airbyte -- -L 8000:localhost:8000 -L 8001:localhost:8001 -N -f
Just visit http://localhost:8000 in your browser and start moving some data!
If you encounter any issues, just connect to our Slack. Our community will help!