Software Engineer - Data Infrastructure - Kafka

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Job Overview

Canonical is seeking a talented Software Engineer to join our Data Infrastructure team with a focus on Kafka. In this role, you will be responsible for developing and maintaining high-performance data pipelines and backend services that enable our clients to integrate, process, and analyze large volumes of data efficiently. You will work collaboratively with cross-functional teams to deliver robust, scalable, and reliable data solutions, ensuring alignment with the company’s comprehensive technology strategy.

Job Responsibilities

• Design, develop, and optimize data collection and processing solutions using Kafka.
• Implement robust streaming data pipelines and ensure data integrity and quality.
• Collaborate with data scientists and analysts to understand their data needs and provide necessary infrastructure support.
• Troubleshoot and resolve performance issues with data systems and services.
• Contribute to the architectural design of next-generation data infrastructure solutions.
• Ensure best practices in monitoring, logging, and scaling of data systems.
• Participate in code reviews, unit testing, and integration testing processes.
• Document software designs, code, and technical processes for future reference.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in software development and engineering.
• Strong knowledge and experience with Apache Kafka and distributed systems.
• Proficiency in programming languages such as Java, Scala, or Python.
• Experience with cloud platforms (AWS, Azure, Google Cloud) and containerization (Docker, Kubernetes).
• Familiarity with data storage technologies (SQL, NoSQL) and data warehousing solutions.
• Understanding of data processing frameworks like Apache Spark or Flink is an advantage.
• Excellent problem-solving skills and the ability to work well under pressure.

Technologies & Tools

This position involves working with a variety of technologies and tools, including but not limited to Apache Kafka, Apache Spark, Docker, Kubernetes, SQL and NoSQL databases, cloud computing platforms (AWS, GCP, Azure), as well as monitoring and logging solutions such as Prometheus, Grafana, and ELK stack.

Ideal Candidates

Ideal candidates for this position will possess a passion for data infrastructure and engineering, coupled with a strong desire to learn and implement cutting-edge technologies. They should exhibit strong analytical skills, attention to detail, and a collaborative mindset, as effective teamwork is essential for success. A proactive approach to problem-solving and a commitment to continuous improvement will be key attributes that align with our team’s values and mission.