Senior Big Data Engineer (Scala and Spark)

We are looking for a Senior/Lead Big Data Engineer to join a distributed team working on a high-impact data platform project for a global client. The goal of the project is to design, build, and maintain robust data pipelines and infrastructure that power reports generated from massive datasets.

You will collaborate with leading Big Data engineers across Europe and internationally, contributing to the full lifecycle of data jobs - from design and implementation to verification and delivery. The team works with the latest versions of Scala and Spark, ensuring the accuracy of statistical outputs consumed by key stakeholders.

While this is primarily a Big Data engineering role, hands-on knowledge of CI/CD and DevOps practices will be a strong advantage, as infrastructure work is an integral part of the day-to-day responsibilities.

Essential functions

• Design, develop, and optimize data pipelines using Scala and Spark
• Build and maintain infrastructure for large-scale data processing and reporting
• Implement and validate algorithms to ensure statistical accuracy for business stakeholders
• Work with distributed computing frameworks and cloud-native architectures
• Collaborate with a globally distributed engineering team on end-to-end data solutions
• Contribute to Data Lake and Data Warehouse implementations
• Support CI/CD pipelines and infrastructure-related tasks as needed

Qualifications

• Strong proficiency in Scala
• Hands-on experience with Spark and Spark Streaming
• Familiarity with Kubernetes, particularly running Spark on Kubernetes
• Experience with Hive and/or Apache Iceberg
• Practical knowledge of Apache Airflow for workflow orchestration
• Solid SQL skills, including SparkSQL - window functions, complex GROUP BY scenarios
• Understanding of distributed computing technologies, patterns, and best practices
• Experience with processing frameworks such as Kafka and Dataframe/SQL API
• Familiarity with database engines: Oracle, PostgreSQL, Teradata, or Cassandra

Would be a plus

• Experience with Apache Flink
• Background in CI/CD pipelines and DevOps tooling
• Experience implementing Data Lakes, Data Warehouses, or analytics platforms
• Hands-on experience with Hadoop
• Familiarity with Claude (Anthropic) or other AI/LLM tooling

We offer

• Opportunity to work on bleeding-edge projects
• Work with a highly motivated and dedicated team
• Competitive salary
• Flexible schedule
• Benefits package - medical insurance, sports
• Corporate social events
• Professional development opportunities
• Well-equipped office

About us

Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.