Senior Data Software Engineer with AWS, LLM, PySpark
We are seeking a highly motivated Senior Data Software Engineer to design, build, and maintain modern cloud-based data platforms while enabling the next generation of AI and Copilot solutions. The ideal candidate will have strong expertise in AWS data ecosystems, PySpark, and experience developing AI-powered solutions using large language models and Generative AI applications.
Responsibilities
- Design and implement scalable ETL/ELT pipelines
- Build and optimize data ingestion frameworks from APIs, databases, SaaS applications, and streaming sources
- Develop data models to support analytics, reporting, AI, and machine learning workloads
- Implement data quality monitoring, lineage, and governance controls
- Support enterprise lakehouse and data warehouse initiatives
- Build serverless and event-driven data processing solutions using AWS Glue, S3, Athena, Redshift, Lambda, EventBridge, and IAM
- Optimize cloud infrastructure costs and performance
- Integrate Generative AI and LLM-based capabilities into enterprise data workflows
- Support Agentic AI and workflow orchestration initiatives
- Implement CI/CD pipelines and automate deployments using Azure DevOps and GitHub Actions
- Support Infrastructure as Code using Terraform or Bicep
- Monitor solution health and performance
Requirements
- 3+ years of working experience with AWS data technologies
- 1+ year of working experience with Generative AI, LLMs, or Copilot Studio
- Experience in Agile product delivery teams
- Proficiency in Python, SQL, PySpark, and Spark SQL
- Expertise in AWS Glue, S3, Athena, Redshift, Lambda, and IAM
- Familiarity with Bedrock
- Knowledge of prompt engineering, RAG (Retrieval-Augmented Generation), and LLM integration
- Background in working with SQL Server and PostgreSQL
- English proficiency at B2 level or higher
Nice to have
- Familiarity with Azure Data Factory, Azure Synapse, and Azure Databricks
- Knowledge of Azure SQL, Azure Data Lake Gen2, and Microsoft Fabric
- Familiarity with Azure OpenAI and Azure Functions
- Skills in Snowflake and Cosmos DB
- Background in Healthcare, Life Sciences, MedTech, or Pharmaceutical industries
- Experience integrating enterprise applications such as SAP, Salesforce, or ServiceNow
- Exposure to Microsoft Fabric and Azure AI Foundry
