Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

Canonical is seeking a highly skilled Software Engineer specializing in Data Infrastructure with a focus on Kafka. In this role, you will be responsible for designing, developing, and maintaining data pipelines and infrastructure that facilitate seamless data flow across various applications and services. You will work closely with cross-functional teams to implement robust solutions that not only ensure reliable data delivery but also enhance data quality and accessibility. Your expertise will be vital in driving efficiency and scalability of data processes within our organization.

Job Responsibilities

• Design and implement scalable data pipelines using Kafka and other data technologies.
• Collaborate with data engineers and scientists to define data requirements and architecture.
• Monitor and troubleshoot performance issues within the data infrastructure.
• Develop and maintain documentation for data architectures, data models, and data workflows.
• Ensure data integrity and security throughout the data lifecycle.
• Stay up-to-date with emerging data technologies and best practices to continuously improve data solutions.
• Participate in code reviews and contribute to team development standards.

Qualifications

• Bachelor’s degree in Computer Science, Software Engineering, or a related field.
• Proven experience with Apache Kafka and streaming technologies.
• Strong proficiency in programming languages such as Java, Scala, or Python.
• Experience with data modeling, ETL processes, and data warehousing concepts.
• Knowledge of cloud platforms (AWS, Azure, GCP) and their data services.
• Familiarity with containerization technologies such as Docker and Kubernetes.
• Excellent analytical and problem-solving skills with a strong attention to detail.

Technologies & Tools

This position will utilize a combination of advanced technologies and tools, including but not limited to Apache Kafka for stream processing, SQL and NoSQL databases for data storage, message brokers, and container orchestration tools like Kubernetes. Familiarity with data visualization and monitoring tools will also be beneficial for ensuring robust data observability.

Ideal Candidates

The ideal candidate for this position will possess not only strong technical skills but also excellent communication abilities and a collaborative mindset. They should be proactive in identifying challenges and proposing innovative solutions, demonstrating a passion for continuous learning and improvement. A strong sense of ownership and accountability will be key in successfully managing data infrastructure projects and delivering high-quality results within a fast-paced environment.