Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

Canonical is seeking a dedicated and skilled Software Engineer specializing in Data Infrastructure, particularly with a focus on Kafka technology. In this role, you will be responsible for the design, implementation, and optimization of data infrastructure solutions that facilitate the processing and transmission of high-volume data streams within the organization. You will collaborate with cross-functional teams to ensure the seamless integration of Kafka into our existing systems, driving improvements in data handling and QoS while contributing to the overall enhancement of Canonical's operational capabilities.

Job Responsibilities

• Design, develop, and maintain robust data infrastructure solutions utilizing Kafka.
• Collaborate with data architects and product teams to create scalable data pipelines.
• Monitor and optimize performance of the data systems to ensure low latency and high throughput.
• Implement best practices for data governance and compliance across data handling processes.
• Debug and troubleshoot data processing issues to minimize downtime.
• Conduct performance tests and tuning to enhance system reliability and efficiency.
• Prepare and present technical documentation and architecture guides for team reference.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or related field.
• Proven experience of at least 3 years in software engineering with a focus on data infrastructure technologies.
• Hands-on experience with Kafka and familiarity with its ecosystem (e.g., Kafka Streams, KSQL).
• Strong knowledge of data modeling, ETL processes, and data warehousing solutions.
• Proficient in programming languages such as Java, Scala, or Python.
• Experience with cloud services (AWS, Azure, or GCP) and containerization technologies (Docker, Kubernetes).
• Excellent problem-solving skills and the ability to work collaboratively in a team-oriented environment.

Technologies & Tools

The position primarily revolves around the use of Apache Kafka for real-time data processing. Familiarity with complementary technologies such as Apache Flink, Apache Nifi, and traditional databases like PostgreSQL or MongoDB is advantageous. The role also involves working with cloud platforms such as AWS or Azure, utilizing Docker for containerization, and employing orchestration tools like Kubernetes for managing application deployment and scaling.

Ideal Candidates

The ideal candidate is a highly motivated individual with a strong technical background and a passion for data engineering. They are proactive, adaptable, and possess excellent communication skills which enable them to articulate complex technical concepts to non-technical stakeholders. A successful candidate is detail-oriented, capable of working independently as well as within a collaborative team environment, and demonstrates a genuine interest in staying current with industry trends and technological advancements.