ShyftLabs

Staff Data Engineer

ShyftLabs

Overview

Role involves implementing and optimizing enterprise data pipelines for Fortune 500 companies.

Ideal candidate should have 6+ years of experience in data engineering with expertise in Databricks and AWS services.

hybridseniorfull-timeEnglishDatabricksAWSApache Spark

Locations

  • Canada, Ontario, Toronto

Requirements

  • Bachelor's or Master's degree required
  • 6+ years of experience in data engineering
  • Expert-level experience with Databricks
  • Hands-on experience with Active Batch or similar platforms
  • Deep knowledge of AWS data services
  • Proven experience with CDC implementations
  • Strong background in Delta Live Tables and streaming processing
  • Understanding of Apache Spark optimization

Responsibilities

  • Implement and optimize data pipeline architecture
  • Build and maintain Databricks job orchestration solutions
  • Implement data ingestion frameworks
  • Troubleshoot pipeline reliability issues
  • Build observability and monitoring solutions
  • Implement near real-time ingestion patterns
  • Build and maintain data quality frameworks
  • Lead technical workshops

Benefits

  • Competitive salary
  • Strong healthcare insurance and benefits
  • Extensive learning and development resources