Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

As a Software Engineer specializing in Data Infrastructure at Canonical, you will play a crucial role in designing and implementing scalable data processing solutions utilizing Apache Kafka. You will collaborate closely with cross-functional teams to ensure robust data pipelines and contribute to enhancing system performance and reliability. Your expertise will directly affect the efficiency of data flow within the organization, empowering data-driven decisions and facilitating innovation.

Job Responsibilities

• Design, develop, and maintain data pipelines using Apache Kafka.
• Optimize performance and scalability of data processing systems.
• Collaborate with data scientists and analysts to understand data needs and enable data accessibility.
• Implement monitoring and alerting solutions to ensure data availability.
• Document system architecture, processes, and best practices for future reference.
• Participate in code reviews and contribute to a culture of continuous improvement.
• Stay up-to-date with emerging technologies and trends in data infrastructure.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience with Apache Kafka and distributed systems.
• Strong programming skills in languages such as Java, Scala, or Python.
• Experience with cloud platforms (e.g., AWS, GCP, Azure).
• Understanding of data modeling, ETL processes, and data warehousing concepts.
• Familiarity with database technologies such as SQL, NoSQL, or graph databases.
• Excellent problem-solving skills and the ability to work collaboratively.

Technologies & Tools

The Software Engineer - Data Infrastructure position primarily involves working with Apache Kafka for stream processing. Familiarity with messaging systems, data storage solutions, and cloud services is crucial. Other technologies and tools include Apache Spark for batch processing, monitoring tools such as Prometheus and Grafana, as well as programming environments for Python and Java development.

Ideal Candidates

Ideal candidates for the Software Engineer - Data Infrastructure role are individuals who possess a combination of technical expertise and strong interpersonal skills. They should demonstrate a proactive mindset towards problem-solving and possess a keen interest in building efficient data solutions. A collaborative spirit, adaptability to changing technologies, and a commitment to quality are essential characteristics that will help candidates excel in this fast-paced environment.