Senior Data Engineer - Snowflake & Airflow

Quantori·Удалённо·Удалённо·3д. назад

Design, develop, and maintain production data pipelines using Python, Apache Airflow, Snowflake, and Amazon S3
Design and evolve scalable data architectures, including Bronze, Silver, and Gold data layers
Develop data models and implement incremental loading, historization, versioning, and reusable transformation patterns
Write and optimize complex SQL transformations, views, stored procedures, dynamic tables, streams, tasks, and scheduled jobs
Integrate data from relational databases, APIs, object storage, and file-based sources
Implement data-quality checks, validation, reconciliation, lineage, monitoring, and robust error-handling processes
Manage Snowflake access controls, roles, permissions, service accounts, and deployment across development, staging, and production environments
Maintain CI/CD pipelines for Snowflake objects, Airflow DAGs, and Python code using Git and automated testing
Monitor and troubleshoot production pipelines and proactively improve reliability and performance
Collaborate with data, analytics, engineering, and other technical teams to translate business and technical requirements into reliable data products
Create technical documentation and specifications following established quality and compliance standards

5+ years of experience in Data Engineering, including at least 3 years working with production data platforms
Strong hands-on experience with Snowflake, including Data modeling, RBAC and access management, Warehouse management, Query profiling and performance optimization, Stored procedures
Advanced SQL skills, including complex joins, CTEs, window functions, incremental processing, and query optimization
Strong Python skills for ETL, database integration, automation, error handling, and testing
Production experience with Apache Airflow, including DAG development, dependencies, sensors, task groups, retries, backfills, and operational support
Experience working with relational databases, object storage, APIs, and file-based data sources
Strong understanding of modern data architectures, including medallion and dimensional modeling, slowly changing dimensions, natural and surrogate keys, and historization
Experience with Git, pull requests, automated testing, and CI/CD for data pipelines and database code
Strong problem-solving skills and the ability to work independently in a production environment
Snowflake and Apache Airflow are core requirements for this role; experience only with alternative data warehouses or orchestration tools would not be sufficient

Experience working in regulated or validated environments
Background in life sciences, healthcare, clinical, laboratory, genomic, or bioinformatics data
Experience working with LIMS or other scientific/clinical data systems
Familiarity with OMOP or other standardized clinical data models
Experience with data-consumption and visualization tools such as Tableau or Streamlit
Familiarity with scientific or computational workflows and tools such as Nextflow or similar workflow orchestration platforms
SnowPro certification
Experience with AWS Managed Workflows for Apache Airflow (MWAA)