Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

As a Software Engineer specializing in Data Infrastructure with a focus on Kafka at Canonical, you will be responsible for designing, implementing, and maintaining robust data pipelines across various applications and services. Your role will involve collaborating with cross-functional teams to enhance data processing capabilities and ensure high availability and performance of data flows. You will leverage your expertise in data architecture and stream processing to improve data handling and infrastructure reliability.

Job Responsibilities

• Design, develop, and optimize scalable data pipelines using Kafka and related technologies.
• Collaborate with data scientists and application developers to provide efficient data solutions that fulfill analytical needs.
• Monitor, troubleshoot and improve the performance of existing data infrastructure.
• Implement data governance principles ensuring data quality, security, and compliance across all platforms.
• Utilize best practices in data modeling and build efficient ETL processes.
• Participate in code reviews and provide constructive feedback to peers.
• Stay current with industry trends and advancements in data engineering and propose innovative solutions.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Proven experience in software development, with a strong emphasis on data infrastructure and stream processing.
• Hands-on experience with Apache Kafka, including its ecosystem components such as Kafka Streams and Kafka Connect.
• Proficiency in programming languages such as Java, Scala, or Python.
• Strong understanding of data structures, algorithms, and database systems.
• Experience with cloud platforms (e.g., AWS, Google Cloud) and data storage solutions.
• Familiarity with Docker and Kubernetes for containerization and orchestration.
• Excellent problem-solving skills and ability to work independently as well as part of a team.

Technologies & Tools

In this position, you will work with various technologies and tools essential for data infrastructure development, including Apache Kafka, Apache Spark, NoSQL databases (e.g., MongoDB, Cassandra), SQL databases, Git for version control, and cloud services such as AWS or Google Cloud Platform. You'll also utilize Docker and Kubernetes for containerization and orchestration, contributing to streamlined DevOps practices.

Ideal Candidates

The ideal candidate for this position is a proactive and analytical thinker who thrives in a fast-paced environment. You possess strong communication skills, enabling effective collaboration with diverse teams. A genuine passion for data engineering and an eagerness to explore innovative data solutions will set you apart. In addition, your ability to adapt to evolving technologies and your commitment to continuous learning will be essential to your success at Canonical.