Senior Data Engineer

About the Position

We are seeking a skilled Data Engineer with experience in Databricks, Snowflake, SQL Server, and cloud technologies. In this role, you will support and manage the organization’s data environment, ensuring reliable data availability and quality. You will serve as a central point of contact for data projects and requests across the organization. The ideal candidate is an independent problem solver with strong communication skills and the ability to work effectively in a rapidly changing environment. The estimated salary range for this position is USD 150,000 to 180,000 per year. This role involves four in-office days per week.

Responsibilities

• Manage Databricks instances, including onboarding datasets, permissions, security, and governance.
• Set up centralized, automated pipelines to import various data sources into the data warehouse.
• Set up automated processes for data refreshes, data validation, and monitoring in the data warehouse.
• Manage day-to-day reporting and data requests.
• Collaborate on data policy and governance.
• Ensure data quality and easy data availability for end users.
• Be the central point of contact for organization-wide data requests and projects.

Requirements

• 3+ years of strong experience in Databricks management.
• 3+ years of experience working with integrated services on Azure.
• 3+ years of experience with object-oriented programming in Python.
• 1+ year of experience with agentic coding assistants.
• Strong experience in pipeline creation, security, governance, and Change Data Capture (CDC).
• Demonstrated ability to independently diagnose and resolve pipeline and data quality issues.
• Experience integrating and ingesting data from external APIs.
• Experience deploying production-ready data assets with minimal oversight.
• Experience with Git-based workflows and CI/CD (e.g., GitHub Actions).
• Experience designing data monitoring and validation frameworks.
• Proficiency in database design, optimization, and performance tuning.
• Experience in data normalization and compression techniques to optimize data size and performance.
• Ensure data integrity, security, and performance in the modernized database environment.
• Should be comfortable working in a rapidly changing environment with ambiguous requirements.
• Familiarity with orchestration tools.
• Strong communication and interpersonal skills, with the ability to communicate with business users.
• Must be able to work independently with minimal guidance.
• Experience with financial data, such as trade data, reference data, position data, etc., to build functional calculation modules, reports, and dashboards.
• Familiarity with reporting tools such as Tableau and Sigma is preferred.