Principal AI Data Engineer

We're looking for a Principal AI Data Engineer to join our team in London, United Kingdom in a hybrid working mode. In this role, you will design and implement advanced AI and GenAI solutions that deliver real business impact. You will work across architecture, engineering and research to build data and AI-enabled platforms, ensuring performance and scalable solutions that accelerate digital transformation for enterprise environments.

Responsibilities

• Develop and evaluate AI/GenAI/AgenticAI prototypes using tools such as Copilot Studio, AI Foundry, Copilot Analyst Agent, Mosaic AI, Genie, AgentBricks and MLflow
• Build and optimize Retrieval-Augmented Generation (RAG) systems including embedding model selection, prompt engineering and traceable evaluation
• Design and deploy basic AI agents using frameworks such as LangChain, AutoGen and smolagents
• Deploy models and applications leveraging Azure OpenAI, Azure AI Foundry, Databricks Mosaic Gateway and containerization with Docker
• Follow DevOps best practices including CI/CD pipelines, testing, linting and GitHub workflows
• Write modular reusable Python code using OOP design patterns and frameworks such as Pydantic and PyTorch
• Emphasize reproducibility, open-source practices and performance tuning in end-to-end delivery
• Operate within agile teams, contributing to sprint planning and collaborative reviews
• Build scalable AI workflows and architectures using tools like Databricks, Genie and LangChain
• Lead architectural planning and backlog creation to ensure long-term reliability and maintainability

Requirements

• Bachelor’s or Master’s degree in Statistics, Mathematics, Econometrics or similar discipline
• 8+ years of experience in data science and AI projects
• Deep understanding of large language models (GPT, Llama, Claude, Mistral) and their reasoning capabilities
• Strong experience with Databricks including DLT, Delta Lake and Unity Catalog governance
• Solid knowledge of streaming technologies such as Spark Structured Streaming and Autoloader
• Proficiency in programming with Python, SQL or Scala
• Expertise in data modeling, ETL/ELT processes and data architecture
• Good understanding of DevOps practices including Git standards and CI/CD automation
• Experience working in Azure cloud environments
• Familiarity with GenAI evaluation frameworks, benchmarking and testing methodologies
• Excellent problem-solving, communication and stakeholder management skills

Nice to have

• Understanding of enterprise AI governance models
• Hands-on experience with Microsoft Copilot, AI Foundry, Databricks MosaicAI and AgentBricks
• Strong decision-making capabilities and conflict resolution skills
• Proven ability to identify and mitigate risks in data-centric projects
• Familiarity with commercial operations in trading and supply environments