Senior Data Integration Engineer
We are actively seeking an experienced Senior Data Integration Engineer to join our team. In this role, you will contribute to the development of cutting-edge data integration pipelines and solutions, playing a pivotal part in shaping the future of our data-driven initiatives.
Responsibilities
- Design, develop, test, deploy, maintain, and enhance data integration pipelines and solutions
- Utilize AWS Glue, Apache Spark, and Databricks to create efficient and scalable data integration processes
- Create and optimize complex SQL queries and database structures for seamless data manipulation
- Collaborate with cross-functional teams to ensure the success of data integration projects
- Contribute to the continuous improvement of data integration processes and tools
- Troubleshoot and resolve data integration issues, ensuring the reliability and accuracy of data pipelines
Requirements
- At least 3 years of relevant work experience in data integration and pipeline development
- Strong expertise in AWS Glue, Apache Spark, and Databricks for seamless data integration and processing
- Proficiency in Python, particularly for data integration purposes, demonstrated through real-life experience
- Solid understanding of SQL for advanced data querying and manipulation
- Hands-on experience with Amazon Kinesis, Amazon S3, and other AWS cloud services for data integration
- Familiarity with source control systems such as GitLab and continuous integration tools like Jenkins for efficient pipeline management
- Fluent English language skills at an Upper-Intermediate level
