Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Software Engineer - Data Infrastructure - Kafka at Canonical will play a pivotal role in designing, developing, and maintaining robust data infrastructure leveraging Kafka. This position entails optimizing the performance and reliability of data pipelines while ensuring seamless data integration and transformation processes. The candidate will collaborate closely with cross-functional teams to implement scalable solutions that enhance data accessibility and usability across organizational platforms.

Job Responsibilities

• Design, implement, and maintain Kafka-based data pipelines ensuring optimal performance and reliability.
• Collaborate with data engineers and software developers to integrate data from various sources.
• Monitor and troubleshoot data flow issues, ensuring minimal downtime and maintaining data integrity.
• Develop and maintain documentation for data systems and infrastructure.
• Perform code reviews and provide constructive feedback to peers to enhance code quality.
• Stay current with emerging technologies and assess their applicability to improve existing processes.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience as a Software Engineer with a focus on data infrastructure.
• Strong understanding of Apache Kafka and experience with its ecosystem.
• Proficiency in programming languages such as Java, Scala, or Python.
• Experience with cloud platforms and data storage solutions (e.g., AWS, Azure, GCP).
• Familiarity with containerization technologies such as Docker and orchestration tools like Kubernetes.
• Excellent problem-solving skills and the ability to work in a team-oriented environment.

Technologies & Tools

The successful candidate will be working with a variety of technologies and tools, including but not limited to Apache Kafka, Confluent Kafka, Kubernetes, Docker, AWS (or similar cloud platforms), and various data processing frameworks. Familiarity with monitoring tools such as Prometheus or Grafana would also be advantageous.

Ideal Candidates

Ideal candidates for the Software Engineer - Data Infrastructure - Kafka position possess a strong technical background complemented by excellent analytical skills. They demonstrate an eagerness to learn and adapt to new technologies and methodologies. Effective communication abilities, both verbal and written, are crucial, as the role involves collaboration with different teams. A proactive approach to problem-solving, attention to detail, and the ability to thrive in a dynamic, fast-paced environment are additional qualities that make an ideal fit for Canonical.