Senior MLOps Developer with AWS AgentCore Runtime
About the Position
We are looking for a Senior MLOps Developer to support enterprise grade lifecycle management for LLM agents and AI systems. In this role, you will work on versioning, CI/CD workflows, promotion controls, and artifact management across development, QA, and production environments.
About the Project
The AI Platform project is centered on building a reliable versioning model for LLM agents as immutable deployment units. It supports controlled release workflows, evaluation based promotion gates, and rollback patterns to help maintain quality and consistency across environments.
Responsibilities
- Develop immutable versioning workflows for agent container images, prompt configuration, tool bindings, and policy associations as a single deployment unit.
- Design and maintain CI/CD pipelines with evaluation gates for AI systems before promotion to production.
- Implement agent promotion workflows across development, QA, and production environments.
- Support rollback patterns for agent versions and validate that restored versions continue to work as expected.
- Manage AWS ECR image practices including ARM64 tagging, lifecycle policies, and image signing for agent immutability.
- Integrate agent versions with a registry and maintain metadata such as evaluation results and promotion status.
- Collaborate on endpoint management patterns for default and named runtime endpoints.
Requirements
- 5+ years of MLOps or DevOps engineering experience with AI or ML systems.
- Experience with versioning and lifecycle management for LLM agents or ML models.
- Experience with immutable artifact versioning at enterprise scale using ECR or model registries.
- Experience designing CI/CD pipelines with quality gates for AI systems.
Nice to Have
- Hands on experience with AWS AgentCore Runtime versioning.
- Experience with agent evaluation frameworks such as Langfuse, LangSmith, or PromptLayer.
- Experience with multi environment agent promotion patterns.
- Experience with agent version comparison and regression detection.
Technologies
AWS AgentCore Runtime, AWS ECR, Langfuse, LangSmith, PromptLayer
