Regular Data Engineer

EPAM·Bulgaria·Офис·3 мес. назад

We are looking for a motivated Data Engineer to join our growing Data Engineering team.

You will work as part of one of our Platform Teams, developing and maintaining data pipelines in Databricks with PySpark on Microsoft Azure for the AI Factory team. This role offers the opportunity to work with modern cloud and big data technologies, contributing to reliable and scalable data platforms that support innovation across the organization.

Responsibilities

Develop and maintain data pipelines in Databricks with PySpark on Azure
Support the design and implementation of cloud-based analytical solutions using Big Data and NoSQL technologies
Assist in building and maintaining data lakes and warehouses to ensure reliability and performance
Participate in the development of ETL/ELT workflows to collect, clean, and structure data
Help implement data quality, lineage, and monitoring frameworks
Collaborate with ML and analytics teams to deliver clean, production-ready datasets
Participate in code reviews and follow technical standards and best practices
Work with CI/CD methodologies in data engineering workflows using tools like Jenkins or GitLab CI/CD
Collaborate with architects, technical leads, and cross-functional teams to deliver solutions aligned with requirements
Engage with stakeholders to understand processes and ensure deliverable alignment

Requirements

2+ years of experience in Data Engineering or a related field
Proficiency in Python and PySpark
Experience with Databricks and Microsoft Azure cloud services
Familiarity with software version control tools (e.g., GitHub, Git)
Exposure to CI/CD frameworks such as Jenkins, Concourse, or GitLab CI/CD
Ability to build reliable and scalable data solutions
Strong problem-solving and analytical skills, with effective communication and teamwork abilities