Airflow, Airbnb's brainchild, is an open-source data orchestration tool that allows you to programmatically schedule jobs in order to extract, transform, or load (ETL) data. Since Airflow's workflows are written in Python as DAGs (Directed Acyclic Graphs) they allow for complex computation, scalability, and maintainability unlike cron jobs or other scheduling tools. Learning and implementing Airflow was an incredibly arduous journey since Airflow is a relatively new and growing technology. I had to figure out how deploy Airflow on a large scale (with the help of Kubernetes!) and develop an entirely automated system.

​

After several weeks, I was finally able to successfully implement Airflow. I taught my teammates all the interesting things (including Airflow, Docker, k8s, and MORE) I learned! Here's just a taste of the five week course I designed :) I had a blast organizing, documenting, and sharing knowledge in a fun and understandable way. Also, my slides just so happen to be super cute!