0

I am building a data pipeline with Airbyte, PostgreSQL and dbt. PostgreSQL and DBT I can easily set up via my main docker-compose.yml but with Airbyte I am not sure. Airbyte itself is a multi-container app so it has it's own docker-compose.yml.

To deploy Airbyte, I follow the steps here and run it with the script `./run-ab-platform.sh. I am not sure though how to integrate this into the main docker-compose.yml.

Current docker-compose.yml structure:

version: "3.8"

services:
  db:
    container_name: postgres
    image: postgres:latest
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=mysecretpassword
      - POSTGRES_DB=training
    ports:
      - '5432:5432'
    volumes:
      - ./postgres:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 60s
      timeout: 30s
      retries: 5
   
  dbt:
    image: fishtownanalytics/dbt:latest
    environment:
      DBT_PROFILES_DIR: /dbt/profile
    volumes:
      - ./my_dbt_profiles:/dbt/profile
    depends_on:
      - postgres

I already tried to use build or extend but can't seem to get it right.

0 Answers0