Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

Canonical is seeking a highly skilled Software Engineer with a focus on Data Infrastructure, specializing in Apache Kafka. The successful candidate will play a critical role in building and maintaining robust data pipelines and systems that support our analytics operations and data-driven decision-making. You will collaborate closely with cross-functional teams to design, implement, and optimize scalable data infrastructure solutions, ensuring the reliability and performance of data streams across various platforms.

Job Responsibilities

• Design, implement, and maintain data pipelines utilizing Apache Kafka and associated technologies.
• Optimize data flow and storage solutions to improve processing efficiency and reliability.
• Collaborate with data analysts and engineers to understand data requirements and develop data models.
• Monitoring system performance and troubleshooting issues as they arise.
• Implement best practices for data governance and security.
• Provide technical leadership and mentorship to junior engineers within the team.
• Stay current with industry trends and integrate relevant technologies into the data infrastructure.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• Proven experience with Apache Kafka and stream processing frameworks.
• Strong programming skills in Java, Scala, or Python.
• Experience with cloud-based data services (AWS, Google Cloud, Azure).
• Understanding of data modeling concepts and ETL processes.
• Excellent problem-solving and analytical skills.
• Strong communication skills and ability to collaborate in a team-oriented environment.

Technologies & Tools

The position will primarily involve working with Apache Kafka for real-time data streaming along with tools such as Apache Spark, Hadoop, and relational databases. Familiarity with containerization tools like Docker and orchestration platforms such as Kubernetes will also be beneficial, as will experience with CI/CD pipelines and data visualization technologies.

Ideal Candidates

The ideal candidates for this role are innovative and proactive problem-solvers who are passionate about data infrastructure and engineering. They should possess a robust analytical mindset and demonstrate the ability to adapt to rapidly changing environments while delivering high-quality work. Effective communicators with strong interpersonal skills who can work well within a diverse team will thrive in this role and contribute to Canonical's mission of delivering exceptional data-driven solutions.