Lead Data Software Engineer with Cloud Services and Databricks
We are seeking a Lead Data Software Engineer to own the end-to-end technical delivery of process mining use cases, migrating from Celonis to a Databricks-based data platform. This role combines hands-on engineering with the establishment of team development practices, mentoring of engineers, and enabling scaling across the account. The ideal candidate is proactive and independent, capable of making sound technical decisions and operating effectively with limited architect capacity.
Responsibilities
- Lead the full engineering lifecycle for process mining use cases, from discovery to production support
- Analyze source systems and implement reliable data ingestion, transformation, and validation pipelines
- Design and develop scalable, reusable Databricks components and Databricks Apps
- Lead PoC migrations from Celonis, assess feasibility, and evolve successful solutions into production
- Establish team engineering practices covering development, unit and integration testing, code review, CI/CD, deployment, and monitoring
- Identify technical risks and dependencies, make design decisions, and collaborate with architects and stakeholders
- Apply AI tools across the SDLC, including requirements analysis, coding, testing, code review, documentation, troubleshooting, and knowledge transfer
- Ensure AI-assisted outputs meet quality, security, compliance, confidentiality, and maintainability standards
- Support engineers through code reviews, documentation, reusable patterns, and knowledge-transfer sessions
- Provide technical input to requirements, acceptance criteria, and delivery planning
Requirements
- 5+ years of hands-on experience with Python and data engineering
- Experience designing and implementing data ingestion and transformation pipelines
- Practical experience with Databricks, data modeling, data quality, and source-system integration
- Knowledge of, and ideally practical experience building and deploying, Databricks Apps
- Experience delivering data or analytics use cases through the complete development lifecycle
- Familiarity with AI-native ways of working and AI tools such as GitHub Copilot, Databricks Genie, and Claude Code
- Experience establishing engineering processes, CI/CD, testing, code review, and release practices
- Demonstrable hands-on experience using AI tools across the SDLC
- Ability to critically review and validate AI-generated code, tests, designs, and documentation
- Experience mentoring engineers and enabling knowledge sharing
- Proactive, independent working style with strong communication and stakeholder-management skills
- Awareness of data governance, security, compliance, data lineage, and privacy requirements
- Proficiency in English at a B2+ level
