Senior Data Software Engineer

We are looking for a Senior Data Software Engineer to design and deliver reusable data-sharing adapters across a cloud lakehouse and external analytics platforms, with a strong focus on governed access and reliable pipelines. You will collaborate with engineers to build modular integrations and demonstrate AI-assisted development in daily work.

Responsibilities

• Design a UniForm lakehouse write layer with dual-format metadata (Delta and Iceberg) for multi-consumer access
• Build and validate GCS-to-BigQuery ingestion pipeline patterns for structured operational datasets
• Implement CDC pipelines using Kafka to support real-time and near-real-time lakehouse updates
• Develop dependency-aware bookkeeping and data lineage tracking patterns across data pipelines
• Engineer modular, version-controlled adapter code that is reusable across new data source integrations
• Configure Snowflake external table definitions and enable governed access via Horizon catalog metadata
• Implement and certify Delta Sharing adapters for zero-copy data sharing to Databricks consumers
• Configure Delta Sharing endpoints, registrations, and sharing agreement management
• Validate end-to-end freshness, sharing latency, and SLA compliance for external data sharing flows
• Implement connector registry entries, RBAC, and tenant-scoped authorization for all data-out paths
• Add metering hooks aligned with billing requirements for governed external data flows
• Document integration patterns and operational steps for reuse across additional data products

Requirements

• 3+ years of experience in data software engineering with Python for data pipeline development
• Experience with Google Cloud BigQuery in advanced usage and optimization
• Experience with data lakehouse table formats including Apache Iceberg and Delta Lake
• Hands-on experience with Databricks integrations and governed data access patterns
• Hands-on experience with Snowflake and external table access for lakehouse data
• Strong knowledge of Kafka and CDC patterns for real-time and near-real-time ingestion
• Strong architecture skills in data lake design, modular adapter development, and version control practices
• Working proficiency with AI-assisted development tools such as Claude Code, GitHub Copilot, or Cursor
• Strong documentation skills for reusable integration patterns and operational runbooks
• Upper-Intermediate English (B2) proficiency for technical collaboration and written communication

Nice to have

• Apache Spark experience for validating shared reads and pipeline patterns
• Databricks Unity Catalog experience for governed metadata and access control
• Delta Lake expertise for sharing and interoperability patterns
• Gen AI Assisted Development experience with measurable workflow improvements
• Snowflake Horizon Catalog experience for metadata governance and access controls