Senior Applied AI Engineer (LLMs, Azure)
We are seeking a Senior Applied AI Engineer to join our AI platform team, focusing on leveraging large language models (LLMs) and Azure technologies for advanced document understanding and data extraction.
You will collaborate with cross-functional experts to design, optimize and validate prompt strategies, ensuring high-quality, secure and scalable AI solutions.
Responsibilities
- Design and optimize advanced prompts for document understanding and data extraction use cases
- Collaborate with business stakeholders to understand document structures, semantics and key-value relationships
- Create PoCs using Python to validate prompt effectiveness and iterate quickly
- Maintain a prompt library with a focus on quality, reusability and compliance with security and privacy guidelines
- Perform prompt validation, stress testing and failover scenario modeling for different LLMs
- Analyze LLM output quality, fine-tune prompts and propose model-specific prompt strategies
- Work with ML engineers on fine-tuning initiatives to improve model accuracy for specific document domains
- Document best practices, prompt strategies and experiment results in a structured format
- Identify and mitigate risks related to prompt injection, hallucination or other LLM-specific vulnerabilities
Requirements
- Proven experience with at least 2 LLM production projects designing and deploying prompts in real-world applications
- Excellent understanding of prompt structure and contexts
- Proficiency in Python with experience in LLM APIs, prompt engineering frameworks and data manipulation tools such as Pandas
- Experience with at least one fine-tuning or retrieval-augmented generation (RAG) pipeline implementation
- Familiarity with prompt evaluation tools and metrics such as BLEU, ROUGE and semantic similarity, as well as human-in-the-loop validation
- Understanding of LLM-related security concerns including prompt injection, output filtering, API limitations and output structuring
- Excellent communication skills and ability to translate complex document requirements into actionable prompt logic
- Strong analytical mindset with attention to both high-level design and low-level output consistency
Nice to have
- Experience with vector databases and retrieval-augmented generation
- Exposure to tools like LangChain, LlamaIndex or similar
- Prior work with OCR, NLP or intelligent document processing (IDP) pipelines
