Senior Data Software Engineer/ Airflow, Snowflake, AI, dbt
We are seeking a Senior Data Software Engineer to implement monitoring, incident management, and operational capabilities for an automated Data Platform Monitoring solution. In this role, you will design and maintain robust monitoring rules, incident automation, and operator health views to ensure the reliability and observability of data products and pipelines.
Responsibilities
- Implement monitoring rules for data pipeline health, freshness, volume, run outcomes, and other agreed monitoring objectives
- Develop data processing and transformation logic for monitoring events and operational data
- Build automated Jira incident creation based on detected monitoring events
- Ensure incidents contain the required information, including component, region, DAG/job, run, timestamp, detecting check, error reference, and severity
- Support incident recurrence handling, stakeholder notification, and monitoring of incident status
- Develop and maintain the operator workspace/health view for monitoring data products and pipeline status
- Enable integration of monitoring data with Snowflake and other components of the solution
- Participate in testing monitoring rules and incident automation against real and historical failures
- Troubleshoot issues, improve monitoring logic, and support solution handover
Requirements
- 3+ years of experience in data engineering roles
- Expertise in Apache Airflow, Snowflake, and dbt
- Proficiency in building and maintaining data pipelines, transformation logic, and operational data flows
- Background in implementing monitoring, observability, and alerting for data platforms
- Skills in incident automation and integration with tools such as Jira
- Familiarity with applying AI capabilities within data platform or monitoring solutions
- Understanding of pipeline health metrics, including freshness, volume, and run outcomes
- Capability to troubleshoot complex data platform issues and drive continuous improvement
- English proficiency at an Upper-Intermediate level (B2) or higher
