Lead DevOps/MLOps Engineer

We are looking for a Lead DevOps/MLOps Engineer to join our ML Engineering team and help build out and harden our on-premises, GPU-backed ML platform.

The goal is to enable multiple ML and LLM use cases to run simultaneously, reliably and with low latency, while scaling the platform to a wider user base. You will work within a 25-person Data Science Department that has grown an in-house agentic platform from a small pilot to roughly 500 employees, with further rollout currently limited by compute capacity. Since all solutions handle confidential documents and sensitive data, the platform runs fully on-premises across 12 (and growing) NVIDIA H200 GPU servers, two Kubernetes clusters, and a stack that includes MLflow, Ray, and an Agent Gateway.

Responsibilities

• Build out and harden the on-premise, GPU-backed ML platform to support multiple concurrent ML and LLM workloads
• Ensure reliability, low latency, and scalability as the user base expands beyond the current ~500 employees
• Mature the GPU environment to production-grade standards, addressing daily bottlenecks and resource contention
• Manage and evolve the GPU infrastructure across 12+ NVIDIA H200 servers, including integration with Kubernetes where appropriate
• Operate and improve the two Kubernetes clusters (production and sandbox) along with platform services such as MLflow and the Agent Gateway
• Support self-hosted LLM deployments for internal, confidential data use cases
• Maintain and extend the Ray environment shared by the Data Science and Deep Learning teams for training workloads
• Enhance monitoring and observability using NVIDIA DCGM exporter, Prometheus, and Grafana
• Collaborate closely with the Traditional ML, Deep Learning, and Data Analytics teams to align infrastructure with evolving use cases

Requirements

• 5+ years of experience in DevOps/MLOps engineering roles
• At least 1 year of relevant leadership experience
• Hands-on experience running GPU workloads on Kubernetes or OpenShift in production, ideally on-premises
• Practical experience deploying and troubleshooting the NVIDIA GPU Operator with a clear grasp of what each component does
• Understanding of how GPUs differ from CPUs as a scheduled resource, including memory limits, utilisation semantics, and the limitations of default Kubernetes scheduling
• Knowledge of GPU sharing and isolation methods such as MIG, time-slicing and MPS, and their trade-offs for memory and fault isolation
• Solid Linux fundamentals including cgroups, namespaces, and systemd resource controls for process-level resource management
• Skills in Prometheus, Grafana and NVIDIA DCGM, including capturing GPU metrics at process or pod level
• Working knowledge of the CUDA stack covering drivers, CUDA toolkit, container runtime, and version compatibility
• Experience deploying with Helm and operating MLOps tools such as MLflow and Airflow on Kubernetes
• English proficiency at an Upper-Intermediate level (B2) or higher

Nice to have

• Familiarity with LLM serving engines such as vLLM, SGLang or NVIDIA Triton
• Experience with Ray/KubeRay and GPU-aware schedulers such as Kueue or Volcano
• Hands-on work with H100/H200-class hardware, GitLab CI and Docker
• Background in Machine Learning and Python