SDE 3 - Data Engineer
Role Description:<\/span>
<\/h1>As a SDE 3 \- Data Engineer, you will play a crucial role in\noverseeing the design, development, and maintenance of our organization's data\narchitecture and infrastructure. You will be responsible for designing and\ndeveloping the architecture for the data platform that ensures the efficient\nand effective processing of large volumes of data, enabling the business to\nmake informed decisions based on reliable and high\-quality data.<\/span>
<\/p>The ideal candidate will have a strong background in data\nengineering, excellent leadership skills, and a proven track record of\nsuccessfully managing complex data projects.<\/span>
<\/p>Responsibilities:<\/span>
<\/h2>● <\/span><\/span><\/span><\/span>Data Architecture and Design:<\/span><\/b>
<\/p>○ <\/span><\/span><\/span><\/span>Design and implement scalable\nand efficient data architectures to support the organization's data processing\nneeds<\/span>
<\/p>○ <\/span><\/span><\/span><\/span>Work closely with\ncross\-functional teams to understand data requirements and ensure that data\nsolutions align with business objectives<\/span>
<\/p>● <\/span><\/span><\/span><\/span>ETL Development:<\/span><\/b>
<\/p>○ <\/span><\/span><\/span><\/span>Oversee the development of\nrobust ETL processes to extract, transform, and load data from various sources\ninto the data warehouse<\/span>
<\/p>○ <\/span><\/span><\/span><\/span>Ensure data quality and\nintegrity throughout the ETL process, implementing best practices for data\ncleansing and validation<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Big Data Technologies:<\/span><\/b>
<\/p>○ <\/span><\/span><\/span><\/span>Stay abreast of emerging trends\nand technologies in big data and analytics, and assess their applicability to\nthe organization's data strategy<\/span>
<\/p>○ <\/span><\/span><\/span><\/span>Implement and optimize big data\ntechnologies to process and analyze large datasets efficiently<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Cloud Integration:<\/span><\/b>
<\/p>○ <\/span><\/span><\/span><\/span>Collaborate with the IT\ninfrastructure team to integrate data engineering solutions with cloud\nplatforms, ensuring scalability, security, and performance<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Performance Monitoring and Optimization:<\/span><\/b>
<\/p>○ <\/span><\/span><\/span><\/span>Implement monitoring tools and\nprocesses to track the performance of data pipelines and proactively address\nany issues<\/span>
<\/p>○ <\/span><\/span><\/span><\/span>Optimize data processing\nworkflows for improved efficiency and resource utilization<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Documentation<\/span><\/b>:<\/span>
<\/p>○ <\/span><\/span><\/span><\/span>Maintain comprehensive\ndocumentation for data engineering processes, data models, and system\narchitecture<\/span>
<\/p>○ <\/span><\/span><\/span><\/span>Ensure that team members follow\ndocumentation standards and best practices.<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Collaboration and Communication:<\/span><\/b>
<\/p>○ <\/span><\/span><\/span><\/span>Collaborate with data\nscientists, analysts, and other stakeholders to understand their data needs and\ndeliver solutions that meet those requirements<\/span>
<\/p>○ <\/span><\/span><\/span><\/span>Communicate effectively with\ntechnical and non\-technical stakeholders, providing updates on project status,\nchallenges, and opportunities.<\/span>
<\/p>
<\/div><\/span>
Requirements<\/h3>Qualifications:<\/span>
<\/h2>● <\/span><\/span><\/span><\/span>Bachelor's or Master's degree\nin Computer Science, Information Technology, or a related field.<\/span>
<\/p>● <\/span><\/span><\/span><\/span>5\-7 years of professional\nexperience in data engineering<\/span>
<\/p>● <\/span><\/span><\/span><\/span>In\-depth knowledge of data\nmodeling, ETL processes, and data warehousing.<\/span>
<\/p>● <\/span><\/span><\/span><\/span>In\-depth knowledge of building\nthe data warehouse using Snowflake<\/b><\/span>
<\/p>● <\/span><\/span><\/span><\/span>Should have experience in data\ningestion, data lakes, data mesh and data governance<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Must have experience in Python programming<\/b><\/span>
<\/p>● <\/span><\/span><\/span><\/span>Strong understanding of big\ndata technologies and frameworks, such as Hadoop, Spark, and Kafka.<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Experience with cloud\nplatforms, such as AWS, Azure, or Google Cloud.<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Familiarity with database\nsystems like SQL, NoSQL, and data pipeline orchestration tools.<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Excellent problem\-solving and\nanalytical skills.<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Strong communication and\ninterpersonal skills.<\/span>
<\/p>● <\/span><\/span><\/span><\/span>Proven ability to work\ncollaboratively in a fast\-paced, dynamic environment.<\/span>
<\/p>
<\/div><\/span>
<\/p>
The ideal candidate will have a strong background in data\nengineering, excellent leadership skills, and a proven track record of\nsuccessfully managing complex data projects.<\/span> ● <\/span><\/span><\/span><\/span>Data Architecture and Design:<\/span><\/b> ○ <\/span><\/span><\/span><\/span>Design and implement scalable\nand efficient data architectures to support the organization's data processing\nneeds<\/span> ○ <\/span><\/span><\/span><\/span>Work closely with\ncross\-functional teams to understand data requirements and ensure that data\nsolutions align with business objectives<\/span> ● <\/span><\/span><\/span><\/span>ETL Development:<\/span><\/b> ○ <\/span><\/span><\/span><\/span>Oversee the development of\nrobust ETL processes to extract, transform, and load data from various sources\ninto the data warehouse<\/span> ○ <\/span><\/span><\/span><\/span>Ensure data quality and\nintegrity throughout the ETL process, implementing best practices for data\ncleansing and validation<\/span> ● <\/span><\/span><\/span><\/span>Big Data Technologies:<\/span><\/b> ○ <\/span><\/span><\/span><\/span>Stay abreast of emerging trends\nand technologies in big data and analytics, and assess their applicability to\nthe organization's data strategy<\/span> ○ <\/span><\/span><\/span><\/span>Implement and optimize big data\ntechnologies to process and analyze large datasets efficiently<\/span> ● <\/span><\/span><\/span><\/span>Cloud Integration:<\/span><\/b> ○ <\/span><\/span><\/span><\/span>Collaborate with the IT\ninfrastructure team to integrate data engineering solutions with cloud\nplatforms, ensuring scalability, security, and performance<\/span> ● <\/span><\/span><\/span><\/span>Performance Monitoring and Optimization:<\/span><\/b> ○ <\/span><\/span><\/span><\/span>Implement monitoring tools and\nprocesses to track the performance of data pipelines and proactively address\nany issues<\/span> ○ <\/span><\/span><\/span><\/span>Optimize data processing\nworkflows for improved efficiency and resource utilization<\/span> ● <\/span><\/span><\/span><\/span>Documentation<\/span><\/b>:<\/span> ○ <\/span><\/span><\/span><\/span>Maintain comprehensive\ndocumentation for data engineering processes, data models, and system\narchitecture<\/span> ○ <\/span><\/span><\/span><\/span>Ensure that team members follow\ndocumentation standards and best practices.<\/span> ● <\/span><\/span><\/span><\/span>Collaboration and Communication:<\/span><\/b> ○ <\/span><\/span><\/span><\/span>Collaborate with data\nscientists, analysts, and other stakeholders to understand their data needs and\ndeliver solutions that meet those requirements<\/span> ○ <\/span><\/span><\/span><\/span>Communicate effectively with\ntechnical and non\-technical stakeholders, providing updates on project status,\nchallenges, and opportunities.<\/span> ● <\/span><\/span><\/span><\/span>Bachelor's or Master's degree\nin Computer Science, Information Technology, or a related field.<\/span> ● <\/span><\/span><\/span><\/span>5\-7 years of professional\nexperience in data engineering<\/span> ● <\/span><\/span><\/span><\/span>In\-depth knowledge of data\nmodeling, ETL processes, and data warehousing.<\/span> ● <\/span><\/span><\/span><\/span>In\-depth knowledge of building\nthe data warehouse using Snowflake<\/b><\/span> ● <\/span><\/span><\/span><\/span>Should have experience in data\ningestion, data lakes, data mesh and data governance<\/span> ● <\/span><\/span><\/span><\/span>Must have experience in Python programming<\/b><\/span> ● <\/span><\/span><\/span><\/span>Strong understanding of big\ndata technologies and frameworks, such as Hadoop, Spark, and Kafka.<\/span> ● <\/span><\/span><\/span><\/span>Experience with cloud\nplatforms, such as AWS, Azure, or Google Cloud.<\/span> ● <\/span><\/span><\/span><\/span>Familiarity with database\nsystems like SQL, NoSQL, and data pipeline orchestration tools.<\/span> ● <\/span><\/span><\/span><\/span>Excellent problem\-solving and\nanalytical skills.<\/span> ● <\/span><\/span><\/span><\/span>Strong communication and\ninterpersonal skills.<\/span> ● <\/span><\/span><\/span><\/span>Proven ability to work\ncollaboratively in a fast\-paced, dynamic environment.<\/span>
<\/p>Responsibilities:<\/span>
<\/h2>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/div><\/span>Requirements<\/h3>
Qualifications:<\/span>
<\/h2>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/p>
<\/div><\/span>