Sr Big Data Infrastructure Engineer (GCP)

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Techaxis, Inc, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Sr Big Data Infrastructure Engineer at Techaxis, Inc. will play a pivotal role in designing, implementing, and optimizing scalable big data solutions on Google Cloud Platform (GCP). This position entails collaborating with cross-functional teams to develop robust data pipelines, ensuring data integrity and security, and leveraging cloud technologies to support data analysis and machine learning initiatives. The ideal candidate will employ their expertise in big data tools and platforms to enhance data infrastructure and drive business intelligence efforts across the organization.

Job Responsibilities

• Design and implement scalable big data infrastructure on GCP, ensuring high availability and performance.
• Develop data ingestion pipelines leveraging technologies such as Apache Beam, Dataflow, and Pub/Sub.
• Monitor and optimize data workflows, ensuring data quality and efficient resource utilization.
• Collaborate with data scientists and analysts to understand data requirements and provide support for analysis and reporting.
• Implement security best practices for data storage and management on GCP.
• Stay abreast of evolving cloud technologies and best practices to recommend enhancements to existing architecture.
• Provide mentorship and guidance to junior engineers and team members on big data technologies and methodologies.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• A minimum of 5 years of experience in big data infrastructure engineering, preferably within a cloud environment.
• Proficient in GCP services such as BigQuery, GCS, Dataflow, and Dataproc.
• Experience with data processing frameworks like Apache Spark and Hive.
• Strong programming skills in Python, Java, or Scala.
• Familiarity with CI/CD practices and DevOps methodologies.
• Excellent problem-solving skills and a detail-oriented mindset.

Technologies & Tools

The Sr Big Data Infrastructure Engineer will utilize an array of technologies and tools including but not limited to Google Cloud Platform (GCP), Apache Beam, Dataflow, BigQuery, Dataproc, Apache Spark, and various ETL tools. Proficiency in cloud security tools and monitoring dashboards is also essential to ensure the reliability and security of data infrastructures.

Ideal Candidates

The ideal candidate for this role possesses a strong analytical mindset, a passion for leveraging data to drive insights, and the ability to effectively collaborate within a team. They should exhibit strong communication skills, demonstrating the capacity to articulate complex technical concepts to non-technical stakeholders. Candidates should also be proactive, adaptable, and eager to learn new technologies, making them well-suited to thrive in a fast-paced and evolving environment.