Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Software Engineer - Data Infrastructure - Kafka at Canonical will be responsible for designing, developing, and deploying robust data infrastructure solutions utilizing Kafka. This role entails collaborating with cross-functional teams to ensure the scalability, reliability, and performance of data pipelines. The ideal candidate will play a key role in enhancing data processing capabilities, contributing to our mission of delivering high-quality software solutions efficiently and effectively.

Job Responsibilities

• Design and implement scalable data pipelines using Apache Kafka to support real-time processing and analytics.
• Collaborate with data scientists and analysts to define data requirements and ensure alignment with business objectives.
• Develop and maintain high-quality, reusable code that adheres to best practices in software engineering.
• Monitor performance and troubleshoot issues within the data infrastructure to enhance its efficiency and reliability.
• Participate in code reviews, mentoring team members, and contributing to a culture of continuous improvement.
• Engage in the evaluation and implementation of new technologies to improve data management processes.
• Document technical designs and data flow processes for knowledge sharing and onboarding purposes.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Proven experience in software development, with a strong emphasis on data infrastructure architecture.
• Hands-on expertise with Apache Kafka and related streaming technologies.
• Proficiency in programming languages such as Java, Scala, or Python.
• Solid understanding of data modeling, ETL processes, and data warehousing concepts.
• Experience with cloud services (AWS, Azure, or Google Cloud) and container orchestration (Docker, Kubernetes).
• Strong problem-solving skills and the ability to troubleshoot complex data pipeline issues.

Technologies & Tools

The position requires proficiency in a variety of technologies and tools including Apache Kafka for streaming data, as well as programming languages such as Java, Python, and Scala. Familiarity with cloud computing platforms like AWS, Azure, or Google Cloud is essential, along with containerization tools such as Docker and Kubernetes. Knowledge of data processing frameworks and databases will also be critical to optimize the data architectures being developed.

Ideal Candidates

Ideal candidates for the Software Engineer - Data Infrastructure - Kafka position will possess a blend of technical expertise and strong communication skills. They will be driven, detail-oriented, and demonstrate a proactive approach to problem-solving and innovation. A collaborative mindset is essential, as the role involves working closely with various stakeholders to deliver effective data solutions. Candidates should exhibit a passion for technology and a commitment to continuous learning and improvement in their craft.