Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

The Software Engineer - Data Infrastructure - Kafka at Canonical plays a vital role in the development, enhancement, and maintenance of the data infrastructure that supports high-scale distributed systems. The successful candidate will engage with multi-functional teams to design and implement scalable solutions using Kafka as a central messaging platform. This role requires a mixture of technical proficiency, problem-solving skills, and the ability to work collaboratively within an agile environment to ensure the stability and efficiency of data streams across various applications.

Job Responsibilities

• Design and develop high-performance data pipelines using Apache Kafka.
• Ensure data integrity and quality while processing large volumes of data.
• Collaborate with cross-functional teams to define data requirements and operationalize insights.
• Monitor and troubleshoot data infrastructure to ensure high availability and performance.
• Implement security practices for data handling and compliance with regulatory requirements.
• Contribute to the architectural design and evolution of the data infrastructure.
• Document technical specifications and operational procedures for future reference.
• Stay updated with industry trends and advancements in data infrastructure technologies.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in software engineering roles with a focus on data infrastructure.
• Strong proficiency in Apache Kafka and its ecosystem (e.g., Kafka Connect, Kafka Streams).
• Experience with programming languages such as Java, Python, or Scala.
• Knowledge of cloud service providers and distributed systems architecture.
• Familiarity with database technologies, both SQL and NoSQL.
• Excellent problem-solving skills and attention to detail.
• Strong communication skills and ability to collaborate effectively within teams.

Technologies & Tools

The role will extensively involve the use of Apache Kafka, as well as related tools in the Apache ecosystem. Familiarity with cloud platforms like AWS, GCP, or Azure will be critical. Additional tools may include Docker and Kubernetes for container orchestration and deployment, as well as frameworks for data processing like Apache Spark.

Ideal Candidates

The ideal candidate for this position is someone who demonstrates strong analytical skills and possesses a passion for data-driven problem-solving. They should have a proactive mindset, being able to take the initiative in exploring new technologies and methodologies. A collaborative spirit and effective interpersonal skills are essential, as this role requires frequent interaction with various teams. Additionally, a continual desire for learning and self-improvement will contribute to both personal success and the accomplishment of team objectives.