Senior Python Engineer with Spark
We are looking for a talented Senior Python Engineer with Spark to join our team working on financial data and analytics. You'll be a crucial player in a team accountable for data integration and transformation, developing models and managing a vast array of structured financial data.
You'll design and build robust software that efficiently processes data through our medallion architecture. We make use of cutting-edge technology with Databricks on Microsoft Azure.
Responsibilities
- Design and build software for processing data through medallion architecture on Databricks in Microsoft Azure
- Apply software engineering processes and principles to data engineering
- Work across the whole software development lifecycle
- Provide mentorship to junior engineers
Requirements
- At least 3 years of experience in Python software development
- Solid understanding of Apache Spark and its applications
- Proficient in Data Modeling and SQL for efficient data processing
- Experience working with Databricks in Microsoft Azure for data processing and analysis
- Ability to work across the whole software development lifecycle, including understanding requirements, contributing to designs, writing code, writing automated tests, providing production support, and helping with deployments
- B2+ English level proficiency
Nice to have
- Experience with Databricks and Delta Lake
