Lead Azure Data Engineer
We are seeking a Lead Azure Data Engineer to join our software, system, and test engineering team.
The role focuses on building and maintaining resilient data pipelines and processing systems for a large user base. Your contributions will ensure our data solutions align with business demands and maintain high industry standards. You will coordinate with architects, tech leads, and team members to deliver on product objectives. If you bring expertise in Azure DevOps, Databricks, and designing scalable data solutions, we encourage you to apply and make an impact through innovative projects.
Responsibilities
- Develop, construct, test, and maintain data pipelines and large-scale customer-facing processing systems
- Setup data framework, maintain and stabilize
- Ensure our products meet business requirements and industry practices
- Recommend ways to improve data reliability, efficiency, and quality
- Collaborate with system architect, tech lead, and team members on product goals
Requirements
- Background in Data Software Engineering with a minimum of 5 years of practical experience
- At least 1 year of relevant leadership experience
- Extensive experience with Azure DevOps CI/CD pipelines, Terraform, and other deployment tools for large-scale infrastructure and application deployment
- Proven expertise in creating and deploying Databricks workflows using Databricks Asset Bundles
- Significant experience with data pipelines on Azure Databricks using Spark
- Demonstrated ability to create and maintain scalable data platforms for a large consumer base
- Proficiency in Python programming
- Excellent written and verbal communication skills in English (B2+ level)
Nice to have
- Experience in creating preliminary prototypes for new product functionalities
- Background in designing data set processes for governance, production, and data modeling
- Familiarity with multiple programming languages and tools for seamless system integration
