Sr Data Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Horizontal Talent, so some details may not match this job. The employer's own page has the real description.

Job Overview

As a Senior Data Engineer at Horizontal Talent, you will play a pivotal role in designing, implementing, and maintaining robust data processing systems. You will collaborate closely with data scientists, analysts, and other stakeholders to facilitate the effective collection, storage, and analysis of large datasets. Your expertise will be crucial in optimizing data pipelines and ensuring data integrity while supporting the development of analytics solutions that drive business insights and decision-making.

Job Responsibilities

• Design and develop scalable data pipelines and ETL processes to ensure efficient data flow.
• Optimize data processing and storage solutions for performance and reliability.
• Collaborate with data stakeholders to understand data requirements and deliver robust solutions.
• Implement data governance measures to ensure data quality and compliance.
• Monitor and troubleshoot data infrastructure, ensuring high availability and performance.
• Work with data modeling techniques to structure and organize large datasets effectively.
• Document data workflows, architectures, and processes for future reference and team usage.
• Mentor junior data engineers and promote best practices within the team.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred.
• 5+ years of experience in data engineering or a similar role.
• Proficient in programming languages such as Python, Java, or Scala.
• Strong knowledge of SQL and experience with data modeling techniques.
• Experience with data lake and data warehouse solutions, particularly AWS or Azure services.
• Familiarity with big data technologies, such as Hadoop, Spark, or Kafka.
• Solid understanding of data governance, quality assurance, and security protocols.
• Excellent problem-solving skills and the ability to work under tight deadlines.

Technologies & Tools

The Senior Data Engineer will utilize a variety of technologies and tools including but not limited to Python, SQL, AWS (S3, Redshift, Glue), Azure, Spark, Kafka, Docker, and Kubernetes. Familiarity with data visualization tools such as Tableau or Power BI, as well as version control systems like Git, will also be beneficial in managing data workflows effectively.

Ideal Candidates

Ideal candidates for the Senior Data Engineer position possess strong analytical and quantitative skills, enabling them to tackle complex data challenges effectively. They exhibit a proactive attitude and a commitment to continuous improvement and innovation. Exceptional communication skills are essential to collaborate with cross-functional teams and explain technical concepts to non-technical stakeholders. The ability to work independently while also being a valuable team player is crucial for success in this role. Finally, a genuine passion for data and technology is key to thriving in the dynamic environment at Horizontal Talent.