Senior Data Software Engineer
We are looking for a Senior Data Software Engineer with experience in using Spark and ideally the Databricks platform to help us build new features around data ingestion, preparation, transformation and exchange related to the modeling process.
Responsibilities
- Creation of pipelines to ingest and transform client exposure and model result data
- Proposal of an innovative solution and proper architecture to allow efficient data transformation and exchange with an external vendor modeling platform
- Evaluation and continuous improvement of the performance of the pipelines
- Review of the work of colleagues and provision of feedback
Requirements
- Minimum 3 years of professional experience in software development
- Strong background in big data engineering development with PySpark
- Expertise in Databricks and the cloud-based lakehouse architecture
- Working experience in an Agile environment (SCRUM)
- Good communication skills
- Upper-Intermediate English language proficiency (B2)
Nice to have
- Expertise in SQL Server management (restore/backup/convert databases)
- Familiarity with infrastructure as code tools (Terraform/Ansible) in cloud environments
- Knowledge of Azure cloud
- Skills in DevOps
