Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

Canonical is seeking a dedicated and skilled Software Engineer specializing in Data Infrastructure with a focus on Kafka. In this dynamic role, you will be responsible for designing, developing, and maintaining robust data pipelines while ensuring the integrity and efficiency of our data architecture. You will collaborate closely with cross-functional teams to optimize data flow and provide solutions that enhance our data-driven decision-making processes. Your efforts will empower analytics and reporting functionalities across the organization, as you leverage your expertise in big data technologies to meet the diverse needs of our users.

Job Responsibilities

• Design and implement data pipelines to process and transport large volumes of data using Kafka.
• Collaborate with data scientists and analysts to ensure data accessibility and integrity.
• Optimize Kafka configurations and deployments for performance and scalability.
• Conduct system performance tuning, fault tolerance, and resilience improvements.
• Monitor and troubleshoot data infrastructure issues in real-time.
• Document technical designs and operational processes for effective knowledge sharing.
• Engage in code reviews and contribute to the development of best practices.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in data engineering or software engineering with a focus on data infrastructure.
• Strong expertise in Kafka and familiarity with other data streaming technologies.
• Proficient in programming languages such as Java, Scala, or Python.
• Solid understanding of database systems, data modeling, and ETL processes.
• Experience with cloud platforms (such as AWS, GCP, or Azure) is a plus.
• Excellent problem-solving skills and the ability to work in a team-oriented environment.

Technologies & Tools

In this position, the Software Engineer will work extensively with various technologies and tools vital for data infrastructure including, but not limited to, Apache Kafka, Apache Spark, relational and non-relational databases, and cloud platforms such as AWS and Google Cloud. Familiarity with CI/CD tools and data orchestration frameworks would also be beneficial for optimizing data delivery processes and ensuring integration across different systems.

Ideal Candidates

The ideal candidates for this Software Engineer position will possess a strong analytical mindset and an eagerness to tackle complex data challenges. They will have a proactive approach to engineering solutions while maintaining high standards of quality and reliability. Excellent communication skills, a collaborative spirit, and a genuine enthusiasm for continuous learning and innovation are essential traits that will contribute to success within this role. Candidates who demonstrate flexibility and adaptability in fast-paced environments will thrive at Canonical.