Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

As a Software Engineer specializing in Data Infrastructure at Canonical, you will play a pivotal role in designing, building, and maintaining high-performance data systems powered by Kafka. You will collaborate with cross-functional teams to enhance our data architecture, ensuring robust data pipelines that facilitate real-time analytics and insights. The ideal candidate will possess strong technical expertise and a keen understanding of data systems, enabling our organization to leverage data effectively in making strategic business decisions.

Job Responsibilities

• Design and implement scalable data processing solutions using Kafka.
• Collaborate with data scientists and analysts to develop and optimize data pipelines.
• Provide ongoing support and maintenance for existing data infrastructure.
• Monitor system performance and troubleshoot issues to ensure reliability and efficiency.
• Participate in the development of best practices and frameworks for data handling.
• Research and evaluate new data technologies and tools to enhance capabilities.
• Document system architecture, processes, and data flows for reference and onboarding.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Proven experience in software engineering with a focus on data infrastructure.
• Strong programming skills in languages such as Java, Python, or Scala.
• Deep understanding of distributed systems and asynchronous programming.
• Hands-on experience with Apache Kafka and related technologies.
• Familiarity with cloud platforms (e.g., AWS, GCP, Azure) and containerization (Docker, Kubernetes).
• Excellent problem-solving skills and the ability to work in a collaborative team environment.

Technologies & Tools

This position will primarily involve working with Apache Kafka for data streaming, alongside additional technologies such as Apache Spark, relational and NoSQL databases, and cloud-based data storage solutions. You will also utilize various software development tools and frameworks to create robust data pipelines and ensure effective data processing workflows.

Ideal Candidates

The ideal candidate for this role will possess not only strong technical skills but also the ability to communicate complex ideas effectively to non-technical stakeholders. A proactive mindset combined with a passion for problem-solving is essential, as is the ability to thrive in a fast-paced, dynamic environment. Candidates should demonstrate a commitment to continuous learning and adaptability to evolving technologies within the data infrastructure landscape.