Senior Software Engineer - Data Platform

You will design, build, and operate foundational data services including cloud data warehouse and data lake storage, orchestration with Airflow, batch processing with Spark and SQL, and streaming with Kafka. You will build and maintain data integration SDKs and self-service applications to enable users to manage and troubleshoot data pipelines. You will design services for end-to-end data security and observability, managing access controls, tracking data quality, cataloging datasets and lineage, and auditing usage. You will translate functional requests from analysts, machine learning, and security and compliance into reusable scalable patterns and assemble data microservices into larger data platforms.

Responsibilities

• Design build and operate foundational data services including data warehouse data lake orchestration batch processing streaming query federation caching time-series and graph databases and real-time aggregation stores
• Build and maintain data integration and process SDKs for internal services and product teams
• Design and build self-service applications to enable users to manage and troubleshoot data pipelines
• Design and build services for end-to-end data security and observability including access controls data quality cataloging lineage and usage auditing
• Translate requests from data analysts machine learning and security and compliance into reusable scalable patterns and assemble data microservices into platforms

Requirements

• 5+ years of software engineering experience
• Strong backend development skills in Python Go or Java
• Experience working with data systems and data pipelines
• Familiarity with design patterns such as scale-out caching key/value and columnar
• Experience using SQL Python Airflow and BI tools to analyze data for operational insights
• Experience responsibly using generative AI tools and copilots with human-in-the-loop practices