Senior Data Engineer (Kafka & Snowflake)

We are seeking a Senior Data Engineer (Kafka & Snowflake) to join a brand-new regulatory reporting team being built from scratch for a global payments technology company. In this role, you will work alongside a small group of engineers and the client’s team to build reliable streaming pipelines on Kafka and reporting models in Snowflake, ensure the data behind regulatory reports is accurate, auditable and reconciled, and help define how the team works and how its architecture takes shape from day one.

Essential functions

• Streaming Pipelines: Design and build Kafka-based data flows with Kafka Streams and/or Kafka Connect, choosing the right delivery guarantees, from at-least-once to exactly-once with idempotent producers and transactions.
• Resilient Event Processing: Implement error handling with retries, dead-letter queues and reprocessing, preserve message ordering, and manage schema evolution through Schema Registry with Avro or Protobuf.
• Reporting Data in Snowflake: Model facts, dimensions and SCD for regulatory reporting, and build ingestion and processing with Streams & Tasks, Snowpipe and Time Travel.
• Data Accuracy & Reconciliation: Implement reconciliation, validation and deduplication so that every reported figure can be trusted and traced.
• Performance & Monitoring: Monitor and tune consumer lag and throughput, and optimise Snowflake queries through execution plans, clustering keys, pruning and warehouse sizing.
• Shaping a New Team: Help define processes and architecture from scratch in a newly formed team.

Qualifications

• Commercial experience as a data or software engineer building production streaming and data pipelines.
• Kafka Expertise: Strong knowledge of Kafka architecture (topics, partitions, consumer groups, offsets, replication), delivery guarantees, Kafka Streams and/or Kafka Connect, and Schema Registry with Avro or Protobuf.
• Advanced SQL in Snowflake: Complex queries with window functions, CTEs, aggregations and large-volume joins, plus hands-on query optimisation.
• Snowflake Platform: Practical experience with Streams & Tasks, Snowpipe and Time Travel.
• Reporting Data Modelling & Quality: Experience modelling data for reporting (facts, dimensions, SCD) and implementing reconciliation, validation and deduplication.
• Ownership & Communication: Independent and proactive, comfortable in a newly formed team, with good communication in English.

Would be a plus

• Apache Flink: Experience with the DataStream API or Flink SQL, stateful processing, checkpointing, event time, watermarks and windowing.
• Java Ecosystem: Java 11/17+, Spring Boot and Spring Kafka, with testing in JUnit, Mockito and Testcontainers.
• Platform & Delivery: Kubernetes (deployments, Helm, scaling), CI/CD, Git and cloud experience.
• Regulatory & Financial Reporting: Experience in banking or payments reporting and a solid understanding of data auditability, lineage and accuracy requirements.

We offer

• Opportunity to work on bleeding-edge projects
• Work with a highly motivated and dedicated team
• Competitive salary
• Flexible schedule
• Benefits package - medical insurance, sports
• Corporate social events
• Professional development opportunities
• Well-equipped office

About us

Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.