Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Software Engineer - Data Infrastructure - Kafka position at Canonical involves developing and maintaining robust data infrastructure solutions using Kafka and other associated technologies. The successful candidate will be responsible for ensuring the reliability, scalability, and performance of large-scale data systems. This role encompasses collaboration with cross-functional teams to design and implement efficient data processing pipelines and contributes to the overall data strategy of the organization.

Job Responsibilities

• Design and implement data processing pipelines using Apache Kafka to support real-time data streaming.
• Ensure the reliability and performance of data infrastructures, troubleshooting and resolving issues as they arise.
• Collaborate with data scientists, analysts, and engineers to define data requirements and architecture specifications.
• Participate in code reviews and contribute to best practices in software development processes.
• Develop and maintain documentation for data systems and processes.
• Monitor and evaluate the performance of the data systems continuously, optimizing where necessary.
• Stay updated with emerging technologies and propose enhancements to the existing data architecture.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in software engineering, with a strong focus on data infrastructure and Kafka.
• Proficient in programming languages such as Java, Scala, or Python.
• Experience with stream processing frameworks and big data technologies.
• Knowledge of cloud environments and services, particularly AWS, GCP, or Azure.
• Strong understanding of database management and data modeling.
• Excellent problem-solving skills and ability to work within a team-oriented environment.

Technologies & Tools

This position primarily utilizes a variety of technologies and tools including Apache Kafka for data streaming, Hadoop for large-scale data storage and processing, and programming languages such as Java and Python. Familiarity with cloud services such as AWS and Azure, as well as frameworks like Apache Spark for data processing, is essential. Additionally, tools for monitoring and performance tracking will be employed to ensure the operational integrity of data systems.

Ideal Candidates

The ideal candidate for the Software Engineer - Data Infrastructure - Kafka role at Canonical will possess a blend of strong technical skills and a collaborative mindset. They are expected to be inquisitive, proactive in identifying and resolving issues, and adept at communicating complex concepts to both technical and non-technical audiences. Successful candidates will thrive in a fast-paced environment and be comfortable working closely with teams across various departments to achieve shared goals.