Principal Data Engineer

Job Title: Principal Data Engineer

Experience: 12–18 Years
Location: Bengaluru
Notice Period: Immediate Joiner

Job Summary

We are seeking a highly accomplished Principal Data Engineer / Data Engineering Architect to lead the design and execution of enterprise-wide data architecture and engineering strategies. This strategic leadership role is responsible for defining engineering standards, driving cloud modernization initiatives, building AI-ready data platforms, and partnering with executive stakeholders to enable data-driven digital transformation.

The ideal candidate will possess deep expertise in enterprise data architecture, cloud-native platforms, distributed data processing, and modern data engineering practices. You will play a pivotal role in shaping the organization's long-term data strategy while mentoring engineering leaders and driving innovation across enterprise data ecosystems.

Key Responsibilities

  • Define and execute the enterprise-wide data engineering and architecture strategy aligned with business objectives.
  • Architect scalable, secure, and cloud-native data platforms on Azure and/or AWS.
  • Lead large-scale cloud migration, data modernization, and digital transformation programs.
  • Design and implement AI-ready Lakehouse architectures to support advanced analytics, machine learning, and AI initiatives.
  • Establish enterprise data governance, metadata management, data quality, security, and compliance frameworks.
  • Drive platform engineering, automation, Infrastructure as Code (IaC), CI/CD, and DataOps best practices.
  • Define engineering standards, architectural principles, development frameworks, and governance models across multiple delivery teams.
  • Provide technical leadership and mentorship to engineering managers, architects, and senior data engineers.
  • Collaborate closely with CXOs, Enterprise Architects, Product Leaders, and business stakeholders to define strategic roadmaps and technology investments.
  • Evaluate emerging technologies, cloud services, and modern data engineering tools to drive continuous innovation.
  • Design scalable API and event-driven architectures for enterprise data integration.
  • Ensure platform scalability, reliability, performance, security, and operational excellence across enterprise environments.
  • Lead architecture reviews, technical governance, and solution validation for complex enterprise programs.

Required Qualifications

  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related discipline.
  • 12+ years of progressive experience in Data Engineering, Data Architecture, or Enterprise Data Platform development.
  • Proven experience designing and implementing enterprise-scale cloud data platforms.
  • Extensive expertise in enterprise architecture, cloud migration, and modernization initiatives.
  • Strong knowledge of Lakehouse architecture, distributed computing, and modern data ecosystems.
  • Experience leading multiple engineering teams across complex, large-scale enterprise programs.
  • Demonstrated success in establishing engineering governance, architectural standards, and technology roadmaps.
  • Strong consulting mindset with the ability to influence business and technology decisions.
  • Exceptional leadership, communication, presentation, and executive stakeholder management skills.

Preferred Qualifications

  • Experience designing AI-ready data platforms supporting Generative AI, Machine Learning, and Advanced Analytics.
  • Cloud certifications such as Microsoft Certified: Azure Solutions Architect Expert, Azure Data Engineer Associate, AWS Certified Solutions Architect, or equivalent.
  • Experience with containerization technologies (Docker, Kubernetes) and microservices architecture.
  • Knowledge of data mesh, data fabric, and modern enterprise data architecture patterns.
  • Familiarity with regulatory compliance frameworks and enterprise security best practices.




Requirements

  • Required Technical Skills

    • Expert programming skills in Python and Scala
    • Extensive experience with Apache Spark
    • Deep expertise in Databricks
    • Strong experience with Snowflake
    • Hands-on knowledge of Apache Kafka and event-driven data processing
    • Strong expertise in Microsoft Azure and/or Amazon Web Services (AWS)
    • Experience with:

    • Azure Synapse Analytics

    • Azure Data Lake Storage

    • Cloud-native Data Platforms

    • Enterprise Data Governance and Metadata Management
    • Data Security, Privacy, and Compliance frameworks
    • MLOps and AI/ML Data Platform architecture
    • API Architecture and enterprise integration patterns
    • Event-Driven Architecture and real-time data streaming
    • CI/CD, DevOps, DataOps, and Infrastructure as Code (Terraform or equivalent)