Kafka Engineer-Senior Associate-Data Analytics Managed Services - Operate

Industry/Sector

Not Applicable

Specialism

Managed Services

Management Level

Senior Associate

Job Description & Summary

At PwC, our people in managed services focus on a variety of outsourced solutions and support clients across numerous functions. These individuals help organisations streamline their operations, reduce costs, and improve efficiency by managing key processes and functions on their behalf. They are skilled in project management, technology, and process optimization to deliver high-quality services to clients.

Those in managed service management and strategy at PwC will focus on transitioning and running services, along with managing delivery teams, programmes, commercials, performance and delivery risk. Your work will involve the process of continuous improvement and optimising of the managed services process, tools and services.

Focused on relationships, you are building meaningful client connections, and learning how to manage and inspire others. Navigating increasingly complex situations, you are growing your personal brand, deepening technical expertise and awareness of your strengths. You are expected to anticipate the needs of your teams and clients, and to deliver quality. Embracing increased ambiguity, you are comfortable when the path forward isn’t clear, you ask questions, and you use these moments as opportunities to grow.

Examples of the skills, knowledge, and experiences you need to lead and deliver value at this level include but are not limited to:

• Respond effectively to the diverse perspectives, needs, and feelings of others.
• Use a broad range of tools, methodologies and techniques to generate new ideas and solve problems.
• Use critical thinking to break down complex concepts.
• Understand the broader objectives of your project or role and how your work fits into the overall strategy.
• Develop a deeper understanding of the business context and how it is changing.
• Use reflection to develop self awareness, enhance strengths and address development areas.
• Interpret data to inform insights and recommendations.

Uphold and reinforce professional and technical standards (e.g. refer to specific PwC tax and audit guidance), the Firm's code of conduct, and independence requirements.

BigData Kafka Developer– Senior Associate (6–9 Years) Our Analytics & Insights Managed Services team brings a unique combination of industry expertise, technology, data management and managed‑services experience to create sustained outcomes for our clients and improve business performance. We empower companies to transform their approach to analytics and insights while building your skills in exciting new directions. Have a voice at our table to help design, build, and operate the next generation of data and analytics solutions as an Associate.

Job Requirements and Preferences

Basic Qualifications

Minimum Degree Required:

Bachelor’s Degree in Engineering, Statistics, Mathematics, Computer Science, Data Science, Economics, or a related quantitative field

Minimum Years of Experience

• 2–5 years of professional experience in analytics, data science, or business intelligence roles
• Preferred Qualifications

Degree Preferred:

Master’s Degree in Engineering, Statistics, Data Science, Business Analytics, Economics, or related discipline

Preferred Fields of Study

• Data Analytics/Science, Statistics, Management Information Systems, Economics, Computer Science
• Preferred Knowledge & Skills
• As a Senior Associate in Data Engineering, you’ll design and optimize scalable data pipelines, streaming solutions, and data models across enterprise platforms. You’ll work with Kafka for real-time streaming, leverage Dataiku and Spark on Cloudera for advanced data workflows, and use Python/PySpark and SQL for data transformations. You’ll also mentor junior engineers, enforce best practices, and collaborate with architects and business stakeholders to ensure robust, efficient, and scalable solutions:
• Data Streaming & Real
• Time Processing (Kafka)
• – Design and implement streaming data ingestion pipelines using Kafka producers, consumers, and topics– Manage partitions, consumer groups, offsets, and retention strategies for scalability and fault tolerance– Integrate Kafka with downstream platforms such as Spark, Cloudera, or Snowflake– Implement monitoring, error handling, and recovery strategies for streaming workloads
• Data Engineering with Dataiku & Spark (Cloudera)
• – Build and optimize ETL/ELT workflows in Dataiku and Spark on Cloudera– Develop reusable and modular data pipelines for batch and near real-time workloads– Optimize Spark jobs using partitioning, broadcast joins, and caching for large-scale datasets– Collaborate with platform teams to ensure efficient execution on Cloudera clusters
• Python & PySpark Development
• – Write scalable Python/PySpark scripts for data cleansing, transformation, and enrichment– Create reusable frameworks and libraries to accelerate development across teams– Lead debugging sessions for distributed jobs and perform advanced performance tuning– Mentor associates on PySpark best practices and coding standards
• Advanced SQL Development
• – Write and optimize complex SQL queries, stored procedures, and functions for analytical and operational workloads– Apply indexing, partitioning, and query tuning techniques to improve performance– Lead validation and reconciliation of datasets across systems and pipelines– Standardize SQL coding practices for junior developers
• Data Modeling & Architecture
• – Design and implement conceptual, logical, and physical data models for enterprise solutions– Lead dimensional modeling (star/snowflake schema) and normalization for OLTP/OLAP systems– Document metadata, entity relationships, and hierarchies for consistent use across teams– Partner with architects to align data models with enterprise strategy and governance
• Data Quality & Governance
• – Define validation frameworks and reconciliation strategies for streaming and batch pipelines– Collaborate with governance teams to ensure data lineage, cataloging, and compliance standards– Proactively monitor and resolve data quality issues impacting downstream analytics
• Version Control & CI/CD Practices
• – Apply Git branching strategies and enforce code review best practices– Contribute to CI/CD pipelines for automated testing and deployment of Spark/Dataiku workflows– Ensure code maintainability and consistency across environments
• Collaboration & Agile Leadership
• – Act as a technical lead for associates, reviewing code, workflows, and mentoring on best practices– Participate actively in sprint planning, backlog grooming, and retrospectives– Work closely with business stakeholders to translate requirements into technical solutions
• Documentation & Knowledge Sharing
• – Maintain detailed documentation for Kafka pipelines, Spark/Dataiku workflows, and data models– Conduct technical deep dives and knowledge-sharing sessions with associates and cross-functional teams– Create runbooks and SOPs for recurring production issues and monitoring
• Soft Skills & Leadership Readiness
• – Strong analytical and problem-solving abilities with focus on optimization and scalability– Ability to explain complex data engineering concepts to non-technical stakeholders– Leadership and mentoring skills to coach associates and guide solution delivery– Ownership mindset with accountability for solution quality and stakeholder satisfaction

Travel Requirements

Not Specified

Job Posting End Date