Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

As a Software Engineer specializing in Data Infrastructure at Canonical, you will play a pivotal role in designing, building, and maintaining robust data pipelines and architectures that enhance our system's performance. Your expertise in Kafka and associated technologies will support our commitment to deliver high-performance, scalable, and reliable data solutions. You will collaborate with cross-functional teams, ensuring that our data infrastructure meets the evolving needs of our products while adhering to the highest standards of software excellence.

Job Responsibilities

• Design, implement, and optimize data processing pipelines using Apache Kafka.
• Collaborate with data scientists, product managers, and other engineers to define data requirements and translate them into technical specifications.
• Ensure the reliability, scalability, and security of data infrastructure components.
• Monitor, troubleshoot, and resolve issues related to data flows and transformations.
• Develop best practices and standards for data management and engineering within the organization.
• Conduct code reviews and contribute to the continuous improvement of team processes and practices.
• Stay up-to-date with industry trends and emerging technologies to proactively address potential challenges in data infrastructure.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Minimum of 3 years of experience in software engineering, with a focus on data infrastructure or data engineering.
• Proficiency in Apache Kafka and familiarity with other data streaming and messaging technologies.
• Strong programming skills in languages such as Java, Python, or Scala.
• Experience with database technologies such as SQL and NoSQL solutions.
• Familiarity with containerization technologies (e.g., Docker, Kubernetes) is a plus.
• Excellent problem-solving skills and the ability to work independently as well as in a team environment.

Technologies & Tools

This position will primarily utilize a range of technologies and tools including, but not limited to, Apache Kafka for data streaming, relational databases (such as PostgreSQL), and NoSQL databases (like MongoDB). Additional technologies may involve Apache Spark for data processing, as well as various DevOps tools such as Docker and Kubernetes to support our development and deployment processes.

Ideal Candidates

The ideal candidates for this role will be those who demonstrate a passion for data engineering and a keen interest in optimizing data flow and infrastructure. They should possess strong analytical skills, an innovative mindset, and the ability to adapt to fast-paced environments. A collaborative approach and excellent communication skills are essential, as is a commitment to continuous learning and improvement in the field of software engineering.