Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

Canonical is seeking a talented Software Engineer to join our Data Infrastructure team, specializing in Kafka. In this role, you will be responsible for designing, developing, and maintaining a robust data pipeline that supports our organization's data needs. You will collaborate with cross-functional teams to implement scalable Kafka solutions that enhance data processing and ensure reliability, availability, and performance. Your expertise will contribute to the evolution of our data infrastructure, enabling data-driven decision-making across the company while helping to shape modern technologies and practices.

Job Responsibilities

• Design and implement robust data pipelines using Apache Kafka.
• Ensure high availability, scalability, and performance of data processing workflows.
• Collaborate with data scientists and analysts to understand data requirements and enhance data processing capabilities.
• Monitor and troubleshoot the Kafka ecosystem to maintain optimal performance.
• Write clean, maintainable, and efficient code while adhering to best practices in software development.
• Participate in code reviews, contributing feedback and suggestions for continuous improvement.
• Stay updated on industry trends and emerging technologies relevant to data infrastructure and processing.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in software development with a focus on data infrastructure.
• Proficiency in Apache Kafka and related streaming technologies.
• Strong programming skills in languages such as Java, Scala, or Python.
• Experience with cloud-based services (e.g., AWS, Azure) and architectures.
• Deep understanding of data modeling, ETL processes, and data warehousing.
• Excellent problem-solving skills and the ability to work independently as well as part of a team.

Technologies & Tools

This position will utilize a variety of technologies and tools including, but not limited to, Apache Kafka, Apache Spark, Docker, Kubernetes, and cloud platforms such as AWS and Azure. Familiarity with monitoring tools like Prometheus and Grafana is also advantageous, along with experience in CI/CD tools to enhance deployment efficiency.

Ideal Candidates

Ideal candidates for this position will exhibit strong analytical skills, a proactive approach to problem-solving, and a passion for working with data and advanced technologies. They will possess excellent communication skills, allowing for effective collaboration with diverse teams. A strong commitment to continuous learning and staying abreast of the latest developments in data infrastructure will mark an individual who thrives in this dynamic environment at Canonical.