Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

The Software Engineer - Data Infrastructure - Kafka position at Canonical is central to enhancing our data processing ecosystem. The successful candidate will be responsible for designing, implementing, and maintaining scalable data pipelines using Apache Kafka. This role requires collaboration with cross-functional teams to develop data-driven solutions that meet the needs of our organization and its customers while ensuring performance and reliability across our data infrastructure.

Job Responsibilities

• Design, implement, and optimize data pipelines using Apache Kafka.
• Collaborate with data architects and software engineers to develop scalable data solutions.
• Monitor and troubleshoot data flow and processing issues within the Kafka ecosystem.
• Develop three-way integrations with external data sources and ensure data quality and integrity.
• Document processes, data models, and best practices for data engineering.
• Engage in code reviews and provide mentorship to junior engineers in the team.
• Evaluate existing data architectures and propose enhancements to improve performance.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience with Apache Kafka, including knowledge of stream processing techniques.
• Strong proficiency in programming languages such as Java, Python, or Scala.
• Hands-on experience with data modeling, ETL processes, and data warehousing solutions.
• Familiarity with cloud platforms (AWS, Azure, or GCP) and containerization (Docker, Kubernetes).
• Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.
• Effective communication skills to articulate complex technical concepts to non-technical stakeholders.

Technologies & Tools

The Software Engineer - Data Infrastructure - Kafka will be working with a variety of modern technologies and tools, including but not limited to Apache Kafka, Apache Flink, various relational and NoSQL databases, and cloud services from platforms like AWS or Azure. Proficiency with container orchestration tools, such as Kubernetes, will also be integral to the role, enabling effective deployment and management of data services in scalable environments.

Ideal Candidates

Ideal candidates for this role will be those who demonstrate a strong passion for data processing technologies and possess a deep understanding of distributed systems. They should exhibit a proactive approach to problem-solving, coupled with the ability to work well both independently and as part of a team. A commitment to quality code, along with continuous learning and adaptation to emerging technologies, will also be essential traits that contribute to their success within Canonical.