Sr. Cloud Data Engineer with (Data Fabric - Big Data Processing - Apache Spark) - Remote

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Zodiac Solutions, Inc, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Senior Cloud Data Engineer at Zodiac Solutions, Inc. will play a critical role in designing, implementing, and managing data processing frameworks utilizing Data Fabric architecture, focusing on Big Data Processing. This position will involve extensive use of Apache Spark to develop scalable data solutions that support analytics and machine learning applications. The ideal candidate will collaborate with cross-functional teams to ensure data integrity, availability, and security while driving the adoption of best practices in data engineering across the organization.

Job Responsibilities

• Design and implement ETL processes within Apache Spark to efficiently process large data sets.
• Develop, maintain, and optimize data pipelines and workflows in a cloud environment.
• Ensure data quality and consistency across multiple data sources and platforms.
• Collaborate with data analysts and data scientists to support data-driven decision-making.
• Monitor system performance and troubleshoot data processing issues as they arise.
• Stay up to date with industry trends and best practices in cloud data engineering and related technologies.
• Participate in code reviews and contribute to team knowledge sharing initiatives.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Data Science, or a related field.
• Minimum of 5 years of experience in data engineering or related roles.
• Strong proficiency in Apache Spark and familiarity with data processing frameworks.
• Experience working with cloud platforms such as AWS, Azure, or Google Cloud.
• Proficient in programming languages such as Python, Java, or Scala.
• Solid understanding of database management systems, including SQL and NoSQL technologies.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

The Senior Cloud Data Engineer will utilize various technologies and tools in their role, including but not limited to Apache Spark for big data processing, AWS or Azure for cloud services, SQL and NoSQL databases for data storage and management, and various ETL tools for data integration. Additionally, familiarity with containerization technologies like Docker and orchestration tools such as Kubernetes is advantageous.

Ideal Candidates

The ideal candidate for this position will possess a blend of strong technical acumen and collaboration skills. They should be proactive, adaptable, and capable of thriving in a fast-paced environment. A passion for leveraging data to solve complex problems, an eagerness to learn new technologies, and an ability to communicate effectively with both technical and non-technical stakeholders will make for a successful addition to the Zodiac Solutions team.