Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Software Engineer - Data Infrastructure - Kafka at Canonical plays a crucial role in designing, developing, and maintaining highly scalable data pipelines and architectures. This position emphasizes building robust Kafka-based solutions to facilitate efficient data processing and real-time analytics, ensuring data integrity and performance. The successful candidate will collaborate with cross-functional teams to implement best practices and innovative solutions, driving the company’s data infrastructure to meet the evolving business needs.

Job Responsibilities

• Design, develop, and optimize Kafka-based data pipelines for extensive data processing.
• Ensure data quality and consistency across various data sources.
• Implement monitoring and logging systems to track data flow and system performance.
• Collaborate with data scientists and engineers to define data requirements and architecture.
• Explore and implement new tools and technologies to improve data infrastructure.
• Conduct code reviews and mentor junior engineers, fostering a culture of continuous improvement.
• Participate in on-call rotations to ensure system reliability and uptime.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in software development, specifically with data infrastructure.
• Strong knowledge of Kafka and experience in building distributed systems.
• Proficiency in programming languages such as Java, Scala, or Python.
• Experience with data modeling and database technologies (SQL and NoSQL).
• Familiarity with cloud platforms such as AWS, Google Cloud, or Azure.
• Knowledge of Docker and Kubernetes for container orchestration is a plus.
• Excellent problem-solving skills and the ability to work in a fast-paced environment.

Technologies & Tools

The Software Engineer - Data Infrastructure - Kafka will utilize a variety of technologies and tools to fulfill their responsibilities, including Apache Kafka for stream processing, cloud platforms like AWS or Google Cloud for hosting, relational and NoSQL databases for data storage, and tools such as Docker and Kubernetes for containerization and orchestration. Familiarity with monitoring tools like Grafana or Prometheus may also be part of the role, ensuring seamless operation of data pipelines.

Ideal Candidates

An ideal candidate for this position possesses a strong analytical mindset, a passion for data-driven solutions, and the ability to work collaboratively within a team. They should demonstrate adaptability in a dynamic environment, have a proactive approach to solving complex problems, and show commitment to continuous learning. Excellent communication skills and the ability to mentor and guide team members will further enhance their fit for the role.