Docker Airflow Installation

In this tutorial you will find steps to install Apache Airflow using Docker. You can follow the document and watch the video for more details on the installation.

Not for production

This guide uses the official Apache Airflow Docker Compose quick-start, which is intended for local development and learning. It is not a production deployment. Before running Airflow in production you need additional hardening: secrets management, HTTPS, database backups, monitoring, resource limits, and strong access controls. See Adding DAGs — Docker for deploying DAGs safely once you move beyond a dev setup.

What this setup includes

The official docker-compose.yaml starts these services:

Service	Role
airflow-webserver	Airflow UI (port 8080)
airflow-scheduler	Schedules and triggers DAG runs
airflow-worker	Runs tasks (CeleryExecutor)
postgres	Metadata database
redis	Celery message broker

DAGs, logs, plugins, and config are stored on the host and mounted into the containers.

Tutorial to follow along

Steps for installation of Apache Airflow using Docker

Create a directory for your Airflow Docker files and DAGs. For this demo everything lives in one place:

sudo mkdir -p /app/airflow/docker
sudo chown -R "$USER:$USER" /app
cd /app/airflow/docker

Download the official Docker Compose file. You can pin a specific Airflow version — this guide uses 2.10.4:

wget https://airflow.apache.org/docs/apache-airflow/2.10.4/docker-compose.yaml

The file is saved as docker-compose.yaml (use that name consistently in all commands below).

Recommended changes to docker-compose.yaml

1. Upgrade Postgres to version 16 (for new installs only)

Find the postgres service and change the image tag:

# Before
image: postgres:13

# After
image: postgres:16

caution

Only change the Postgres major version on a fresh install. If you already have data in a Postgres 13 volume, upgrading requires a database migration — do not change the tag blindly.

2. Disable example DAGs

Add this to your .env file (created in the next step) instead of editing the compose file:

AIRFLOW__CORE__LOAD_EXAMPLES=false

Create the folders Airflow expects:

mkdir -p ./dags ./logs ./plugins ./config

Docker needs a consistent user ID on the host to avoid permission issues on dags/, logs/, and plugins/. Create .env with your user ID and the settings above:

cat > .env <<EOF
AIRFLOW_UID=$(id -u)
AIRFLOW__CORE__LOAD_EXAMPLES=false
EOF

Optionally set a custom admin username and password (otherwise the defaults from airflow-init apply — typically airflow / airflow):

echo "_AIRFLOW_WWW_USER_USERNAME=admin" >> .env
echo "_AIRFLOW_WWW_USER_PASSWORD=admin" >> .env

Initialize the database, run migrations, and create the admin user:

docker compose up airflow-init

Start Airflow. Add -d to run containers in the background:

# Foreground (logs in terminal — good for first run)
docker compose up

# Background (typical after everything works)
docker compose up -d

When startup finishes, open the Airflow UI at http://localhost:8080.

Default login (if you did not set custom credentials in .env): username airflow, password airflow.

Stopping and restarting

# Stop containers (keeps data volumes)
docker compose down

# Start again in the background
docker compose up -d

Upgrading Airflow version

1. Stop the stack: docker compose down
2. Download the new version's compose file, e.g. wget -O docker-compose.yaml https://airflow.apache.org/docs/apache-airflow/2.11.0/docker-compose.yaml
3. Re-apply your Postgres and .env settings
4. Run docker compose up airflow-init then docker compose up -d

Always read the Airflow release notes for breaking changes before upgrading.