Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

As a Software Engineer focusing on Data Infrastructure at Canonical, you will play a pivotal role in enhancing and supporting our data processing capabilities, specifically utilizing Kafka. You will be responsible for designing and implementing robust data pipelines to ensure efficient data flow and processing. Your contributions will help drive innovation and performance throughout our engineering teams, ensuring that we leverage data effectively in our products and services.

Job Responsibilities

• Design, develop, and maintain data pipelines using Kafka to facilitate real-time data processing.
• Collaborate with cross-functional teams to gather requirements and deliver scalable data infrastructure solutions.
• Implement data models and architecture best practices to optimize performance and reliability.
• Monitor and troubleshoot data processing systems to ensure high availability and prompt resolution of issues.
• Participate in code reviews and contribute to the overall improvement of the engineering practices.
• Document processes, data flows, and system designs to ensure knowledge sharing within the team.

Qualifications

• Proven experience in software engineering, specifically in data pipeline development.
• Strong proficiency in Kafka and stream processing technologies.
• A solid understanding of data modeling, ETL processes, and data warehousing concepts.
• Fluency in programming languages such as Java, Python, or Scala.
• Experience with container orchestration platforms like Kubernetes is a plus.
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

Technologies & Tools

The Software Engineer - Data Infrastructure role at Canonical involves working with a variety of technologies and tools, including Apache Kafka for stream processing, various data storage solutions (e.g., SQL and NoSQL databases), Docker for containerization, and monitoring tools like Prometheus and Grafana. Familiarity with automation tools and cloud services such as AWS or GCP is also beneficial.

Ideal Candidates

Ideal candidates for this position will possess strong analytical and problem-solving skills, with a keen ability to troubleshoot complex issues. They should demonstrate a proactive approach to learning and adopting new technologies, while also being effective communicators, capable of articulating technical concepts to non-technical stakeholders. A passion for data architecture and a drive to contribute to innovative solutions are essential traits for success in this role.