Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

As a Software Engineer specializing in Data Infrastructure with a focus on Kafka at Canonical, you will be responsible for designing, implementing, and optimizing data pipelines that leverage Kafka for real-time data processing. Your role will involve collaborating with cross-functional teams to build scalable and reliable data infrastructure solutions, ensuring seamless integration of data and fostering data-driven decision-making across the organization. You will play a pivotal part in enhancing our data landscape while contributing to best practices in software development and data management.

Job Responsibilities

• Design and implement robust data pipelines utilizing Apache Kafka to facilitate real-time data processing.
• Develop and maintain data models, ensuring data accuracy, integrity, and availability.
• Collaborate with data scientists and analysts to translate business requirements into scalable solutions.
• Troubleshoot and optimize Kafka clusters and related components for improved performance and reliability.
• Participate in code reviews and provide mentorship to junior engineers to enhance team capabilities.
• Document architecture, design processes, and technical specifications for future reference.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree is a plus.
• Proven experience as a Software Engineer, with a focus on data infrastructure and streaming technologies.
• Strong proficiency in Apache Kafka and related data processing frameworks.
• Solid understanding of data modeling and database management systems.
• Experience with programming languages such as Java, Scala, or Python.
• Familiarity with cloud platforms and containerization (e.g., AWS, Docker, Kubernetes) is advantageous.

Technologies & Tools

The Software Engineer - Data Infrastructure role will primarily utilize Apache Kafka for real-time data streaming, alongside tools like Apache Spark for data processing, relational databases like PostgreSQL, and NoSQL databases such as MongoDB. Familiarity with cloud-based systems on platforms such as AWS or Azure will also be crucial, alongside container orchestration tools like Docker and Kubernetes to ensure a seamless development and deployment environment.

Ideal Candidates

Ideal candidates for this position will exhibit strong analytical thinking and problem-solving abilities, enabling them to navigate complex data challenges effectively. They will demonstrate solid collaboration skills, thriving in a team-oriented environment, while also showing independence in managing their workload. A passion for data and a commitment to best practices in software development will be essential, as will a proactive approach to learning and adapting to new technologies and methodologies. These attributes, combined with technical expertise, will contribute to their success within Canonical’s dynamic work environment.