Software Engineer - Data Infrastructure

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Canonical, so some details may not match this job. The employer's own page has the real description.

Overview

The Software Engineer - Data Infrastructure at Canonical will play a pivotal role in designing, developing, and maintaining robust data infrastructure systems that support scalable and high-performance applications. The engineer will collaborate closely with cross-functional teams to ensure seamless data management and integration, thus enabling data-driven decision-making and insights across the organization. This position requires a deep understanding of data architecture, analytics, and pipeline production, and offers the opportunity to contribute significantly to the company's data-driven culture and technological advancements.

Job Responsibilities

• Design, implement, and optimize scalable data pipelines for processing large datasets.
• Collaborate with data scientists and analysts to understand data requirements and translate them into technical specifications.
• Ensure robust data quality and availability through regular data monitoring and performance tuning.
• Develop and maintain data models and monitoring dashboards to provide insights into system performance.
• Create documentation for data infrastructure processes and participate in code reviews to uphold best practices.
• Work with cloud platforms to enhance data storage and retrieval mechanisms.
• Participate in agile development processes, including sprint planning and retrospectives.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of experience in software engineering focused on data infrastructure.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with data processing frameworks like Apache Spark or Kafka.
• Strong knowledge of SQL and experience with relational and NoSQL databases.
• Familiarity with cloud platforms such as AWS, Google Cloud, or Azure.
• Excellent problem-solving skills and the ability to work in a team-oriented environment.

Technologies & Tools

This position will utilize an array of technologies and tools including, but not limited to, Apache Spark, Kafka, SQL databases (e.g., PostgreSQL, MySQL), NoSQL databases (such as MongoDB), as well as cloud services provided by AWS and Google Cloud. Familiarity with containerization technologies like Docker and orchestration systems such as Kubernetes will also be beneficial in managing our data infrastructure effectively.

Ideal Candidates

The ideal candidate for the Software Engineer - Data Infrastructure position will possess a strong analytical mindset with a keen attention to detail, enabling efficient problem-solving when faced with complex data challenges. A passion for learning new technologies and a proactive approach to improving existing systems will set apart strong applicants. Candidates should excel in communication and collaboration, ensuring effective coordination across teams. A commitment to best practices and data integrity will also be essential for success in this role.