Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

The Software Engineer - Data Infrastructure - Kafka at Canonical will be instrumental in building and maintaining scalable and high-performance data infrastructure that leverages Kafka. This role requires innovation and creativity in developing solutions for data streaming and processing. By collaborating with cross-functional teams, the Software Engineer will ensure efficient data flow and reliability while implementing best practices in software development and architecture. This position demands a deep understanding of distributed systems and a commitment to enhancing data-driven decision-making across the organization.

Job Responsibilities

• Design, implement, and optimize data pipelines using Kafka to ensure high availability and fault tolerance.
• Collaborate with data scientists, analysts, and other engineers to understand data needs and identify solutions.
• Monitor and troubleshoot data infrastructure issues, ensuring performance and scalability of the data systems.
• Maintain documentation of the data architecture and best practices.
• Implement data governance and security practices to protect sensitive information.
• Participate in agile methodologies, including sprint planning, reviews, and retrospectives.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in software development with a focus on data infrastructure.
• Strong expertise in Apache Kafka, including architecture, design, and implementation.
• Proficiency in programming languages such as Java, Scala, or Python.
• Experience with cloud platforms such as AWS, GCP, or Azure.
• Familiarity with containerization technologies like Docker and orchestration tools like Kubernetes.
• Understanding of database technologies, both SQL and NoSQL.

Technologies & Tools

The Software Engineer - Data Infrastructure - Kafka will utilize a variety of technologies and tools including Apache Kafka for real-time data streaming, relational databases like PostgreSQL or MySQL, NoSQL databases such as MongoDB, cloud services including AWS or Google Cloud Platform, and containerization tools like Docker and Kubernetes. Additionally, proficiency with monitoring tools like Prometheus or Grafana may be needed to ensure the health of data systems.

Ideal Candidates

Ideal candidates for the Software Engineer - Data Infrastructure - Kafka role will possess strong analytical and problem-solving skills, with a passion for data architecture and infrastructure. They should be proactive and self-motivated, able to thrive in a fast-paced, collaborative environment. A commitment to continuous learning and adaptation to emerging technologies is essential. Strong communication skills are also necessary to articulate complex data concepts effectively to both technical and non-technical stakeholders.