Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Software Engineer - Data Infrastructure - Kafka at Canonical will be instrumental in designing, implementing, and maintaining robust data infrastructure systems to ensure efficiency and scalability. The role involves a strong focus on Kafka for real-time data streaming applications, contributing to enhancing data accessibility and performance across the organization. The successful candidate will collaborate with cross-functional teams to optimize data pipelines and integrate complex systems, ensuring seamless data flow to support the company's goals.

Job Responsibilities

• Design, develop, and maintain data streaming applications using Kafka.
• Implement data pipelines and workflows to facilitate efficient data processing.
• Collaborate with data scientists and analysts to understand data requirements and optimize data accessibility.
• Monitor and troubleshoot data infrastructure to ensure high availability and reliability.
• Conduct performance tuning and optimization of data applications.
• Document systems architecture and provide technical guidance to team members.
• Stay updated with industry trends and best practices in data infrastructure and Kafka technologies.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in software development, with a strong emphasis on data infrastructure.
• Hands-on experience with Apache Kafka and its ecosystem.
• Proficient in programming languages such as Java, Scala, or Python.
• Strong understanding of data modeling, ETL processes, and data warehousing concepts.
• Experience with distributed systems and microservices architectures.
• Familiarity with cloud computing platforms (e.g., AWS, Azure, GCP) is a plus.

Technologies & Tools

In this position, candidates will work with a variety of advanced technologies and tools, including but not limited to Apache Kafka for data streaming, Apache Spark for big data processing, various databases (SQL and NoSQL), and cloud platforms such as Amazon Web Services (AWS) or Google Cloud Platform (GCP). Additionally, familiarity with DevOps tools for CI/CD pipelines and containerization technologies (e.g., Docker, Kubernetes) will be advantageous.

Ideal Candidates

The ideal candidate for the Software Engineer - Data Infrastructure - Kafka role will exhibit a strong analytical mindset, coupled with innovative problem-solving skills. A proactive approach to tackling challenges and a passion for data-driven decision-making are essential. Candidates should also possess excellent communication skills, enabling collaboration with various teams and stakeholders. Furthermore, a commitment to continuous improvement and a desire to stay abreast of emerging technologies will greatly benefit the dynamic nature of this role.