DevOps Engineer with GCP, Big Query, Airflow, DBT

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Peritus Inc, so some details may not match this job. The employer's own page has the real description.

Job Overview

Peritus Inc. is seeking a proactive and skilled DevOps Engineer with expertise in Google Cloud Platform (GCP), BigQuery, Apache Airflow, and DBT (Data Build Tool). The ideal candidate will work collaboratively with cross-functional teams to architect, deploy, and manage scalable data pipelines, ensuring high availability and performance. This role involves automating operational processes, enhancing data integration solutions, and using best practices in data governance and security. A strong focus will be placed on optimizing workflows and ensuring reliable data delivery to support data analytics and decision-making processes across the organization.

Job Responsibilities

• Design, implement, and manage robust data pipelines utilizing GCP technologies.
• Develop and maintain ETL processes using Apache Airflow and DBT.
• Monitor and optimize data workflows, ensuring high levels of availability and reliability.
• Collaborate with data scientists and analysts to understand data requirements and translate them into scalable solutions.
• Ensure data security, compliance, and governance protocols are followed across all processes.
• Automate operational processes for efficiency and scalability.
• Diagnose and troubleshoot issues within the data pipeline and suggest improvements.
• Provide documentation and train team members on best practices and tools used.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or related field.
• Minimum of 3 years of experience in a DevOps or data engineering role.
• Proficient in GCP, specifically BigQuery, Cloud Storage, and related services.
• Strong experience with Apache Airflow for orchestrating data workflows.
• Hands-on experience with DBT for data transformation.
• Familiarity with containerization technologies such as Docker and Kubernetes.
• Knowledge of scripting languages such as Python or Bash.
• Experience with CI/CD practices and tools.

Technologies & Tools

The DevOps Engineer will primarily work with Google Cloud Platform services, including BigQuery and Cloud Functions, to manage and analyze large datasets. The role will extensively utilize Apache Airflow for orchestrating data workflows and DBT for transforming data in the warehouse. Other technologies may include Docker for containerization, Terraform for infrastructure as code, and monitoring tools such as Grafana or Prometheus to ensure efficient operations.

Ideal Candidates

Ideal candidates for the DevOps Engineer position should be highly analytical and possess strong problem-solving skills. They should demonstrate a thorough understanding of cloud infrastructure and data engineering principles. Excellent communication skills and the ability to collaborate with diverse teams are essential, along with a proactive attitude towards continuous improvement and learning. A candidate will thrive in this role if they are adaptable, detail-oriented, and able to work in a fast-paced, evolving environment.