Senior Python Engineer with Spark

We are looking for a talented Senior Python Engineer with Spark to join our team working on financial data and analytics. You'll be a crucial player in a team accountable for data integration and transformation, developing models and managing a vast array of structured financial data.

You'll design and build robust software that efficiently processes data through our medallion architecture. We make use of cutting-edge technology with Databricks on Microsoft Azure.

Responsibilities

• Design and build software for processing data through medallion architecture on Databricks in Microsoft Azure
• Apply software engineering processes and principles to data engineering
• Work across the whole software development lifecycle
• Provide mentorship to junior engineers

Requirements

• At least 3 years of experience in Python software development
• Solid understanding of Apache Spark and its applications
• Proficient in Data Modeling and SQL for efficient data processing
• Experience working with Databricks in Microsoft Azure for data processing and analysis
• Ability to work across the whole software development lifecycle, including understanding requirements, contributing to designs, writing code, writing automated tests, providing production support, and helping with deployments
• B2+ English level proficiency

Nice to have

• Experience with Databricks and Delta Lake