Lead Apache Airflow Engineer
We are seeking a Lead Apache Airflow Engineer to drive the development and maintenance of Airflow-based workflows, guide technical delivery, and mentor team members while collaborating with cross-functional and distributed teams.
Responsibilities
- Develop new and maintain existing Airflow Operators, Hooks and Sensors
- Integrate new Airflow versions with Cloud Composer
- Fix bugs in Airflow and implement improvements
- Implement tests for Airflow, including Unit Testing and Solution Testing
- Create internal documentation for the work artifacts
- Coordinate development activities within the team and support delivery planning
- Collaborate with cross-functional and distributed teams to support delivery and resolve technical issues
- Support junior team members through knowledge sharing, technical guidance, and code review activities
- Participate in technical discussions with client, prioritization, and coordination of development activities within the team
Requirements
- 7+ years of experience with strong Python programming skills
- Experience with Cloud APIs (REST/RPC)
- Knowledge of DevOps/SysAdmin practices, Linux, and Kubernetes, including building container images
- Understanding of Cloud platforms, especially Google Cloud Platform
- Expertise in Airflow or similar experience in building workflows based on Luigi, Oozie, Argo, Azkaban, or Apache NiFi
- Experience working with open-source communities
- Proficiency in English communication, both spoken and written
Nice to have
- Understanding of cloud architecture, including scalability and IaaS principles
- Google Cloud Certification (e.g., GCP Architect or Data Engineer)
