DE&A - Core - Advanced Data Engineering - Advanced Data Engineering (Other)

We are seeking a Data Engineer to support the development of a scalable, enterprise-wide data platform for a leading insurance client. The role involves building and maintaining batch and real-time data pipelines, enabling self-service analytics (Power BI), and contributing to report rationalization initiatives.

The candidate will work with a modern cloud stack including Snowflake, DBT, AWS Glue, and Amazon Kinesis, ensuring efficient data ingestion, transformation, and optimization across Raw, Silver, and Gold layers.

Key Responsibilities

Data Ingestion & Processing

• Develop and maintain batch data pipelines using AWS Glue
• Build and support real-time data ingestion pipelines using Amazon Kinesis
• Integrate data from multiple insurance systems (policy, claims, billing, third-party sources)

Data Transformation & Modeling

• Implement transformations using DBT across Raw, Silver, and Gold layers
• Create scalable data models to support reporting and analytics use cases
• Ensure consistent, reusable, and well-documented data assets

Data Platform Support

• Work on Snowflake-based data warehouse to manage and optimize data storage and access
• Support medallion architecture implementation (Raw → Silver → Gold)
• Collaborate with senior engineers and architects on platform enhancements

Reporting Enablement & Rationalization

• Support Power BI self-service reporting by delivering curated, high-quality datasets
• Assist in report rationalization efforts, identifying redundant or inefficient reports
• Align data models to business KPIs and insurance domain requirements

Pipeline Optimization

• Monitor and optimize data pipelines for performance, scalability, and cost efficiency
• Troubleshoot data issues, bottlenecks, and failures in pipelines
• Apply best practices in query tuning and data processing efficiency

Data Quality & Governance

• Implement and maintain data validation and quality checks
• Ensure adherence to data governance standards and controls
• Support audit and reconciliation requirements

Required Skills & Experience

Technical Skills

• Hands-on experience with:
  • Snowflake (data warehousing basics, querying, optimization)
  • DBT (transformations, models, testing)
  • AWS Glue (batch ETL pipelines)
  • Amazon Kinesis (real-time streaming)
• Strong proficiency in SQL and basic data modeling concepts
• Programming knowledge in Python (preferred)

Analytics & Reporting

• Exposure to Power BI for data consumption and reporting
• Understanding of data structures for analytics use cases

Optimization & Troubleshooting

• Ability to identify and resolve pipeline performance issues
• Basic understanding of cost optimization techniques in cloud environments

Domain Knowledge (Preferred)

• Experience in Insurance domain (policy, claims, underwriting, billing)
• Familiarity with enterprise data ecosystems and reporting needs in insurance

Experience

• 3–6 years of experience in Data Engineering / ETL Development
• Experience working on cloud-based data platforms

Key Success Metrics

• Reliability and performance of data pipelines
• Data quality and consistency across layers
• Timely delivery of curated datasets for reporting
• Contribution to report rationalization and platform efficiency

Data Engineer

Data Engineer experience