Lead Data Integration Engineer
We are seeking a Lead Data Integration Engineer to build and optimize data pipelines within the AWS ecosystem, enabling seamless integration and robust data reporting for critical decision-making processes.
Responsibilities
- Collaborate with business analysts to understand and implement data requirements
- Design and develop efficient data pipelines using AWS technologies
- Ensure reliability, performance, and scalability of data pipelines
- Troubleshoot and resolve data integration issues promptly
- Implement data mappings, aggregations, and quality checks based on business analyst input
- Maintain clear and comprehensive documentation of integration processes
- Support data asset transition processes using Iceberg
- Facilitate data querying and manipulation with SQL
- Leverage tools like DBT and Airflow for pipeline orchestration and data modeling
- Align data integration processes with compliance standards and best practices
Requirements
- 5+ years of proven experience in data integration and data warehousing
- Proficiency in DBT, Airflow, and SQL for data transformation, orchestration, and querying
- Skills in Python for API development and automation
- Expertise in Iceberg for data asset management and transition
- Knowledge of Snowflake and Power BI for data warehousing and visualization
- Demonstrated ability in data modeling and database performance tuning
- Familiarity with implementing data quality checks and aggregation processes
- Strong problem-solving, analytical, and communication skills
- Effective use of data integration tools in cloud-based environments (AWS)
