Senior Data Engineer
We are seeking a highly skilled Senior Data Engineer to design, develop and maintain robust data solutions for our business needs. The ideal candidate will have expertise in data engineering technologies, a strong coding background and excellent communication skills to collaborate directly with clients.
This position requires working with time shifts and possible late meetings with the onsite team before 10 am PST.
Responsibilities
- Architect and maintain ETL/ELT pipelines, batch/micro-batch processing and streaming systems
- Build and optimize infrastructure for data extraction, transformation and loading using tools like ADF, Spark and Kafka
- Identify and implement internal process improvements to enhance scalability and performance
- Design and implement innovative data services solutions involving Spring Boot, ReactJS and NoSQL where required
- Act as a single point of responsibility for delivery-related matters including escalations and technical oversight
- Lead technical design to ensure a robust, scalable architecture that meets quality standards
- Coordinate across disciplines and stakeholders to ensure seamless delivery
- Ensure adherence to customer processes, methodologies and agile workflows
- Establish risk management strategies during the delivery life cycle
- Drive strategies for improving delivery productivity and efficiency
- Support Data Engineers by providing technical consulting and guidance
- Manage deployment and production support activities for delivered solutions
Requirements
- 5+ years of experience with SQL, Spark and Scala
- Proficiency with Databricks and Spark Streaming technologies
- Knowledge of Kafka for stream processing and data integration pipelines
- Skills in leveraging Azure for data solutions and cloud management
- Strong communication skills with professional-level English aptitude enabling direct client communication
Nice to have
- Familiarity with the Azure ecosystem for advanced data solutions
- Understanding of innovative big data solutions for personalized customer experiences
