Responsibilities:
Maintain our data platform with timely and quality dataPlan and execute data platform expansion to support the company’s growth and analytic needs. Define and extend our internal standards for style, maintenance, and best practices for a high-scale data platformBuild and maintain data pipelines from internal databases and SaaS applicationsWrite maintainable, performant code and create and maintain systems documentationDesire to continually keep up with advancements in data engineering practices.Solve technical problems of the highest scope and complexityImplement the DataOps philosophy in everything you doDesign and develop code to extend the enterprise data modelBuild trust in all interactions and with trusted data developmentCollaborate with Analytics Engineers and Data Analysts to drive efficiencies for their workCollaborate with other functions to ensure data needs are addressed
Requirements:
Python/Scala/JavaData Processing(Apache Spark/Apache Flink/Apache Beam, etc.) and Pipeline orchestration (Apache Airflow, Apache NiFi end, etc)Message Brokers and distributed streaming platformDB (SQL/NoSQL/Open formats)Dev Tools (Git/Docker/Kubernetes/Jira/etc)Infrastructure/Cloud (AWS/GCP/Azure/etc). Data Privacy/SecurityAlgorithms/Data Structure/Data ModelingData Development Principles/ProcessWriting Requirements/DocumentationCI/CD/CD Practical familiarity with DataOps, Data Lake, Data Mart, and DataMesh/Data Hub principles
Nice to have: AWS Redshift, AWS Glue, S3, EMR, DMS, QuickSight, MWAA, etc
