Senior Data Engineer
Responsibilities
- Review mapping definition with data analysts
- Contribute with other data scientists on mapping definition for transformation pipelines
- Contribute to mapping function development and unit testing of these functions
- Contribute to mapping configurations
- Define the design of quality control pipelines, ensuring mapping accuracy and alignment with analytical expectations
- Review and maintain the quality control pipelines with the Data Analyst, Data Domain Lead and Data Governance Lead
- Implement or configure the appropriate tools for the approved quality control pipelines
- Provide the guidelines for introducing Machine Learning and LLM in the transformation pipelines
Requirements
- 10+ years of experience in Data Engineering
- Strong familiarity with the Databricks platform (including Unity Catalog)
- Ability to write robust code with Python
- Proficiency with Spark (PySpark)
- Knowledge of cloud platforms: AWS
- Experience in implementing ML and LLM in production data pipelines
- Ability to collaborate closely with Central European time zone
Nice to have
- Knowledge of Lakehouse concept including experience using Delta Lake optimized storage layer
- Familiarity with SODA is a plus
We offer
- Competitive compensation
- Remote or office work
- Flexible working hours
- Healthcare benefits: medical insurance and paid sick leave
- Continuous education, mentoring, and professional development programs
- A team with an excellent tech expertise
- Certifications paid by the company
