Senior Data DevOps Engineer
We are seeking a Senior Data DevOps Engineer to join our team. We are building an Enterprise AI Gateway from scratch. It is the one door every team in the company goes through to use large language models. It is also how we keep AI under control at this scale: who can use which models, what it costs, and what gets logged. The aim is that it runs itself. Onboarding of teams, agents, and MCP servers, along with keys, permissions, limits, and guardrails, should be handled by automation, not by tickets. Very little of it exists yet, so you will shape it from the very first design decision. The team is small and focused, so decisions do not take long and your work is visible. We are looking for someone who automates by default, and who cares about the governance side of AI as much as the models themselves.
Responsibilities
- Design and build the core architecture of the Enterprise AI Gateway from the ground up
- Automate the onboarding of teams, agents, and MCP servers, including provisioning of keys, permissions, limits, and guardrails
- Develop and maintain infrastructure as code to support scalable, self-service platform operations
- Implement monitoring, logging, and cost-tracking mechanisms to maintain visibility and control over AI usage
- Build guardrails and governance controls to manage which teams and models can access specific resources
- Ensure the reliability, availability, and performance of production systems supporting the gateway
- Collaborate directly with teams using the platform, addressing questions and helping resolve errors
- Continuously improve automation to reduce manual intervention and ticket-based processes
- Evaluate and integrate emerging GenAI and agentic AI patterns, frameworks, and protocols into the platform
- Contribute to key architectural and design decisions as the platform evolves from an early stage
Requirements
- A minimum of 3 years of relevant experience
- Strong Python skills for automation, extensions, and integrations
- Strong SRE skills, with real-world experience keeping production systems healthy
- Proficiency with Git for version control
- Experience with Google Cloud Platform
- Familiarity with LLMOps practices
- Experience with Terraform and Helm for infrastructure automation
- Working knowledge of GenAI/Agentic AI concepts, including relevant patterns, frameworks, and protocols
- Very good communication skills, with the ability to clearly explain technical concepts, since you will interact daily with teams using the platform, answering questions and helping resolve issues
- Excellent English proficiency (B2 level or higher)
Nice to have
- Hands-on experience with Google Vertex AI, particularly endpoints for model serving and Model Armor
- GCP experience with services such as BigQuery, Cloud Run, and IAM
- Experience building AI agents, for example using Google's Agent Development Kit (ADK)
- Experience with AWS Bedrock
- Deep Kubernetes experience, preferably with GKE
- Familiarity with Groovy
- Experience with CI/CD using Jenkins
- Production experience with an AI gateway, such as LiteLLM or EPAM DIAL, highly appreciated
