Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Software Engineer - Data Infrastructure - Kafka role at Canonical involves the design, development, and maintenance of scalable data infrastructure using Kafka. The successful candidate will play a crucial role in enhancing the data pipelines and ensuring the reliability and efficiency of data communication across various systems. By leveraging their expertise in data processing frameworks and messaging systems, they will contribute to high-performance, fault-tolerant data solutions that meet the evolving needs of the organization.

Job Responsibilities

• Design, implement and maintain scalable and robust data infrastructure leveraging Kafka.
• Collaborate with cross-functional teams to optimize data ingestion and processing workflows.
• Develop data solutions that ensure high availability and low-latency data access.
• Monitor and troubleshoot existing data pipelines and provide enhancements as needed.
• Implement best practices for data governance and security across data integrations.
• Participate in code reviews and contribute to the continuous improvement of software engineering processes.
• Stay up-to-date with industry trends and emerging technologies related to data infrastructure.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 3+ years of experience in software engineering, with a focus on data infrastructure and Kafka.
• Strong programming skills in Java, Python, or Scala.
• Experience with messaging systems, particularly Kafka and its ecosystem components.
• Proficiency in database technologies, both SQL and NoSQL.
• Knowledge of cloud platforms such as AWS, Google Cloud, or Azure is a plus.
• Familiarity with containerization and orchestration tools like Docker and Kubernetes.

Technologies & Tools

The position requires proficiency in various technologies and tools critical for data infrastructure management. Candidates will work extensively with Apache Kafka for data streaming, in addition to using related tools such as Apache Zookeeper for managing distributed systems. Familiarity with ETL tools, data modeling technologies, and relational/NoSQL databases will be essential. Furthermore, knowledge of cloud services, particularly AWS or Google Cloud, will significantly enhance performance and productivity in this role.

Ideal Candidates

The ideal candidates for the Software Engineer - Data Infrastructure - Kafka position will possess a strong analytical mindset with the ability to identify and solve complex problems efficiently. They should demonstrate excellent communication skills, enabling effective collaboration with diverse teams. A proactive approach to learning new technologies and adapting to changing requirements will be essential. Moreover, candidates should exhibit a strong commitment to quality and a drive for continuous improvement in their work processes.