Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

Canonical is seeking a talented Software Engineer specializing in Data Infrastructure with a focus on Kafka to join our dynamic team. In this role, you will be responsible for the design, implementation, and maintenance of scalable data pipelines and streaming solutions. You will collaborate with cross-functional teams to optimize data flow and enable real-time processing that supports data-driven decision-making across the organization. Your expertise will be crucial in enhancing our data ecosystem while ensuring reliability, performance, and security.

Job Responsibilities

• Design and develop scalable data pipelines leveraging Apache Kafka.
• Implement and maintain data streaming solutions to support various applications.
• Ensure data integrity and reliability across different systems and processes.
• Collaborate with data scientists and analysts to optimize data workflows.
• Monitor and troubleshoot existing data infrastructure to ensure performance and availability.
• Document technical specifications and best practices for future reference.
• Stay abreast of emerging technologies and industry trends in data processing and architecture.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• Proven experience in software development with a strong emphasis on data pipelines and infrastructure.
• In-depth knowledge of Apache Kafka and similar streaming technologies.
• Proficiency in programming languages such as Java, Python, or Scala.
• Experience with cloud-based data services and containerization technologies like Docker.
• Familiarity with database technologies (SQL and NoSQL) and data warehousing concepts.
• Strong analytical and problem-solving skills with attention to detail.

Technologies & Tools

This position will utilize a range of technologies and tools including, but not limited to, Apache Kafka, Kubernetes, Docker, SQL and NoSQL databases (MongoDB, PostgreSQL), cloud platforms (AWS, Azure), and Python for scripting and automation. Familiarity with tools for monitoring and performance tuning is also advantageous for optimizing the data infrastructure.

Ideal Candidates

The ideal candidate for this role will possess a strong technical background combined with excellent communication skills. They should be highly motivated, proactive in seeking solutions, and able to work effectively within a fast-paced, team-oriented environment. A passion for data-driven methodologies and a commitment to delivering high-quality, maintainable code will set successful candidates apart. Additionally, the ability to translate complex technical concepts to non-technical stakeholders is crucial for collaboration across the organization.