Role Overview
:
We are seeking a Senior Data Engineer with strong developer DNA — someone who codes daily in PySpark, SQL, and Python, builds end-to-end data solutions, and leads design of robust, production-grade data platforms on Microsoft Fabric, Azure Databricks, and ADF. This is a hands-on leadership role for engineers who can think architecturally and code efficiently.
Key Responsibilities:
• Design, develop, and optimize complex data pipelines and lakehouse architectures.
• Lead development of batch and streaming ingestion frameworks using ADF, Databricks, and Event Hubs.
• Write optimized PySpark/SQL transformations for performance, scalability, and cost efficiency.
• Build reusable data frameworks and automation scripts for continuous integration and deployment.
• Establish unit testing, code review, and versioning standards across engineering teams.
• Integrate with Dev Ops CI/CD pipelines for automated releases and environment management.
• Mentor junior engineers on coding practices and architecture design.
• Collaborate with data architects on Fabric-based Lakehouse implementations.
Skills &
Qualifications:
• 6–12 years of strong
hands-on experience
in data engineering and development.
• Expertise in:
o PySpark, SQL, Python, and performance tuning of large-scale transformations.
o Databricks, ADF, ADLS, Fabric Data Pipelines, and Delta Lake.
• Deep understanding of data architecture, schema design, job orchestration, and data governance.
• Strong knowledge of CI/CD (Azure Dev Ops) and infrastructure-as-code for data pipelines.
• Optional:
Exposure to GCP Big Query, Dataproc, or Dataflow as secondary platforms.
Preferred
Certifications:
• Databricks Certified Data Engineer Professional
• Microsoft Certified:
Fabric Analytics Engineer Associate (DP-600)
• Microsoft Certified:
Azure Solutions Architect Expert (AZ-305)
• Optional:
Google Cloud Certified – Professional Data Engineer
