Installation

There are three ways to run the Airflow Operator:

Helm managed Docker container deployment on Kubernetes
As a Docker container
Build from source.

Helm

Helm allows you to download and deploy Stackable operators on Kubernetes and is by far the easiest installation method. First ensure that you have installed the Stackable Operators Helm repository:

$ helm repo add stackable https://repo.stackable.tech/repository/helm-stable/

Then install the Stackable Operator for Apache Airflow

$ helm install airflow-operator stackable/airflow-operator

Helm will deploy the operator in a Kubernetes container and apply the CRDs for the Apache Airflow service. You are now ready to deploy Apache Airflow in Kubernetes.

Docker

This Operator is published as a Docker image:

docker.stackable.tech/stackable/airflow-operator

When installing manually with Docker you will need to install the Stackable CRDs for Apache Airflow in your Kubernetes environment. These are available on the Stackable GitHub repository for this operator.

$ kubectl apply -f airflowcluster.crd.yaml
$ kubectl apply -f init.crd.yaml

To run it straight from Docker you can use this command:

docker run \
    --name airflow-operator \
    --network host \
    --env KUBECONFIG=/home/stackable/.kube/config \
    --mount type=bind,source="$HOME/.kube/config",target="/home/stackable/.kube/config" \
    docker.stackable.tech/stackable/airflow-operator:latest

Building the operator from source

This operator is written in Rust and is developed against the latest stable Rust release (1.58 at the time of writing).

stackable-airflow-operator crd | kubectl apply -f -
stackable-airflow-operator run