Apache Airflow, Data Engineering, Dev Ops

[12/23] Testing: A Cost-Effective Unified Data Platform for Small Organizations

Testing Apache Airflow An important aspect to take care of when writing software is testing. Test-driven development is…

Apache Airflow, Data Engineering

[11/23] Workflow: A Cost-Effective Unified Data Platform for Small Organizations

In order to build data pipelines between our data sources and data warehouse, the workflow of a DAG…

Apache Airflow, Data Engineering

[10/23] Operators: A Cost-Effective Unified Data Platform for Small Organizations

In Airflow, the way of creating a task is by using an operator. Since a DAG describes how…

Apache Airflow, Data Engineering

[9/23] Directed Acyclic Graphs: A Cost-Effective Unified Data Platform for Small Organizations

Directed Acyclic Graphs The most important concept in Airflow is the directed acyclic graph (DAG). A concise definition…

1 2 3 4 5