AWS DevOps Engineer (Data Cataloguing)

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by AI TalentFlow, so some details may not match this job. The employer's own page has the real description.

Job Overview

The AWS DevOps Engineer (Data Cataloguing) at AI TalentFlow is responsible for designing, implementing, and managing data cataloging solutions on AWS. This role requires collaboration with data engineering and analytics teams to ensure seamless integration and governance of data assets across the organization. The ideal candidate will leverage AWS tools and services to build a robust data cataloging infrastructure that enhances data discoverability, accessibility, and security while optimizing workflows and processes.

Job Responsibilities

• Design and implement AWS-based data catalog solutions leveraging services such as AWS Glue, AWS Lake Formation, and Amazon Athena.
• Collaborate with data architects and analysts to understand data requirements and define metadata standards.
• Monitor and manage data catalog performance, ensuring high availability and quick access for users.
• Maintain data governance policies to ensure compliance with legal and regulatory requirements.
• Automate data catalog processes including ingestion, classification, and transformation of data assets.
• Conduct training sessions for stakeholders on effective use of data cataloging tools and best practices.
• Participate in agile development practices including sprint planning and code reviews.
• Stay current with AWS offerings and industry trends, recommending improvements and enhancements to current services.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or related field.
• 3+ years of professional experience in DevOps or software engineering roles.
• Strong proficiency with AWS services, particularly those related to data management and cataloging.
• Hands-on experience with infrastructure as code tools such as AWS CloudFormation or Terraform.
• Familiarity with data governance principles and best practices.
• Experience with CI/CD tools and methodologies.
• Proficient in scripting languages such as Python, Bash or similar.
• Strong analytical and troubleshooting skills with a focus on automation.

Technologies & Tools

The AWS DevOps Engineer will utilize a variety of technologies and tools including but not limited to AWS Glue, AWS Lake Formation, Amazon RDS, Amazon S3, and monitoring tools such as AWS CloudWatch. Familiarity with data processing frameworks like Apache Spark and SQL databases will also be beneficial. Additionally, version control systems such as Git will be essential for managing code and deployments.

Ideal Candidates

Ideal candidates for the AWS DevOps Engineer (Data Cataloguing) position are proactive, detail-oriented individuals with a passion for data and cloud technologies. They should possess strong problem-solving abilities and be adept at collaborating within cross-functional teams. Excellent communication skills and a commitment to continuous learning are crucial, as is a drive to innovate and improve existing processes. Candidates should be able to thrive in a fast-paced environment and adapt quickly to changing requirements.