Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

Canonical is seeking a skilled Software Engineer to join our Data Infrastructure team, focusing on the Apache Kafka ecosystem. In this role, you will be responsible for designing, implementing, and maintaining robust data pipelines, enabling the seamless flow of data across various platforms. As part of a dynamic team, you will contribute to enhancing our data handling capabilities and ensuring data quality and integrity, all while optimizing performance and scalability in line with business objectives.

Job Responsibilities

• Design and implement scalable data ingestion frameworks using Apache Kafka.
• Develop and maintain data streaming applications that ensure efficient data processing.
• Collaborate with cross-functional teams to understand data requirements and improve data architecture.
• Monitor data flow and troubleshoot issues to ensure high availability and performance.
• Implement best practices for data pipeline reliability, metadata management, and data governance.
• Create and maintain documentation for data infrastructure processes and systems.
• Participate in code reviews and provide constructive feedback to enhance code quality.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Strong experience with Apache Kafka and data streaming concepts.
• Proficiency in at least one programming language such as Java, Scala, or Python.
• Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services.
• Understanding of data modeling, data integration, and ETL processes.
• Experience with containerization technologies such as Docker and Kubernetes.
• Excellent problem-solving skills and ability to work in a fast-paced environment.

Technologies & Tools

In this position, you will work extensively with technologies such as Apache Kafka for data streaming, as well as various database systems like PostgreSQL and MongoDB. Additionally, you will utilize cloud technologies and services, alongside tools for monitoring and troubleshooting data pipelines, including Grafana and Prometheus. Familiarity with continuous integration and deployment (CI/CD) tools will also be necessary for efficient software delivery.

Ideal Candidates

Ideal candidates for this position possess a strong analytical mindset and a passion for working with big data technologies. They demonstrate a proactive approach to problem-solving and have a collaborative spirit that fosters teamwork. Excellent communication skills are essential for articulating complex technical information to non-technical stakeholders. Candidates should thrive in a dynamic, fast-paced environment and have a genuine interest in advancing data infrastructure capabilities that drive innovation within the organization.