Lead Data Engineer

Job Title: Lead Data Engineer

Experience: 10–14 Years
Location: Bengaluru
Notice Period: Immediate or 15days

Job Summary

We are seeking an experienced and visionary Lead Data Engineer to lead enterprise-scale data engineering initiatives and drive the design, implementation, and modernization of cloud-native data platforms. The ideal candidate will have extensive experience in defining enterprise data architecture, leading high-performing engineering teams, and delivering scalable data solutions using modern cloud technologies. This role requires strong technical leadership, architectural expertise, and the ability to collaborate with cross-functional teams and business stakeholders.

Key Responsibilities

  • Define and implement enterprise data architecture aligned with business and technology strategies.
  • Lead multiple data engineering teams across complex enterprise projects and cloud transformation initiatives.
  • Design, build, and optimize scalable cloud-native data platforms and modern data ecosystems.
  • Architect and implement Lakehouse solutions using industry best practices.
  • Drive enterprise-wide data governance, data quality, metadata management, and security initiatives.
  • Review solution architecture, technical designs, and code to ensure adherence to engineering standards and best practices.
  • Collaborate with Solution Architects, Product Owners, Business Analysts, and stakeholders to translate business requirements into scalable technical solutions.
  • Ensure platform reliability, scalability, performance, and operational excellence.
  • Champion automation, CI/CD, Infrastructure as Code (IaC), and DataOps best practices.
  • Mentor and coach data engineers, fostering technical excellence and continuous learning.
  • Identify opportunities for process improvements and technology modernization.
  • Provide technical leadership throughout the software development lifecycle, from design through deployment and support.

Required Qualifications

  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related discipline.
  • 10–14 years of experience in Data Engineering, Data Platform Development, or Cloud Data Architecture.
  • Proven experience leading enterprise-scale data engineering programs and cloud migration initiatives.
  • Strong expertise in designing scalable, secure, and high-performance data platforms.
  • Demonstrated experience leading and mentoring technical teams.
  • Strong understanding of data modeling, data warehousing, Lakehouse architecture, and distributed data processing.
  • Excellent analytical, problem-solving, and decision-making skills.
  • Outstanding communication, stakeholder management, and client-facing skills.

Preferred Qualifications

  • Experience with enterprise data governance frameworks and metadata management solutions.
  • Knowledge of real-time streaming and event-driven architectures.
  • Experience with Infrastructure as Code (Terraform), containerization, and orchestration technologies.
  • Azure certifications or cloud architecture certifications are highly desirable.
  • Familiarity with Agile, Scrum, and modern software engineering practices.


Requirements

  • Required Technical Skills

    • Strong programming expertise in Python and Scala
    • Extensive experience with Apache Spark
    • Hands-on experience with Databricks
    • Strong knowledge of Snowflake
    • Experience with Apache Kafka and event-driven data architectures
    • Deep expertise in Microsoft Azure cloud services
    • Hands-on experience with:
    • Azure Data Factory (ADF)

    • Azure Synapse Analytics

    • Azure Data Lake Storage

    • Strong understanding of CI/CD pipelines and DevOps practices
    • Experience implementing DataOps methodologies
    • Knowledge of Terraform or other Infrastructure as Code (IaC) tools is highly preferred