Senior Data Integration Engineer (Databricks)
We are looking for a Senior Data Integration Engineer to join the dynamic Data Practice team and collaborate on EPAM strategic projects.
As a Senior Data Integration Engineer, your key role will be to build data systems and applications that drive decision-making. You will primarily focus on developing and maintaining data pipelines and data-driven solutions in public clouds like AWS or Azure, as well as migrating data from the on-premises data centers to the cloud. Additionally, you will build components for enterprise data platforms.
Responsibilities
- Play a key role in driving forward EPAM strategic data projects
- Design, build, and manage enterprise data solutions in Public Cloud (AWS, Azure)
- Develop components for the enterprise data platform to ensure data accuracy and availability
- Collaborate with data scientists, data architects, and others ensure data needs are met
- Implement secure and auditable data processes to ensure data integrity
- Troubleshoot any data-related issues and strive to improve the quality of data systems
Requirements
- Proven experience as a Data Engineer or similar role for at least 3 years
- Proficiency in SQL and Phyton
- Proven experience with Public Cloud (AWS, Azure), DBMS and ETL/ELT tools
- Hands-on experience with Databricks
- Experience in data acquisition (API calls/FTP downloads), ETL, transformation/normalization (from raw to DB table schema structure), storage (Raw files, Database server), distribution & access (Entitlements for users, build of API’s and access points for data)
- Experience building components for enterprise data platform (Data Warehouses, Operational Data Stores, Access layers with APIs, file extracts, user queries)
- A bachelor's degree in computer science, IT, or a related field
- Strong communication skills
