Remote Sr. Big Data Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by SnapX.ai, so some details may not match this job. The employer's own page has the real description.

Overview

SnapX.ai is seeking a detail-oriented and innovative Remote Sr. Big Data Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and maintaining scalable data architectures that enable robust data processing and analytics. The ideal candidate will work collaboratively with cross-functional teams to transform raw data into actionable insights, ensuring high availability and performance of data pipelines while adhering to industry best practices in data governance and security.

Job Responsibilities

• Design and implement end-to-end big data solutions using distributed computing technologies.
• Develop and manage data pipelines for ingestion, transformation, and storage of large datasets.
• Optimize data processing workflows to improve efficiency and reduce latency.
• Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets.
• Monitor, troubleshoot, and resolve issues related to data processing and system performance.
• Ensure compliance with data governance standards and best practices.
• Document processes, architecture, and changes for future reference and audit purposes.
• Stay updated with the latest big data technologies and trends for continuous improvement.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field. A master's degree is preferred.
• Minimum of 5 years of experience in big data engineering or a related discipline.
• Proficiency in big data technologies such as Hadoop, Spark, or Kafka.
• Strong programming skills in languages such as Python, Java, or Scala.
• Experience with data modeling, ETL processes, and SQL or NoSQL databases.
• Familiarity with data warehousing solutions and cloud platforms (AWS, Google Cloud, Azure).
• Demonstrated ability to work collaboratively in a remote team environment.
• Excellent analytical, problem-solving, and communication skills.

Technologies & Tools

The Remote Sr. Big Data Engineer will work with a variety of technologies and tools integral to big data processing and management. The primary tools include Apache Hadoop for distributed storage, Apache Spark for data processing, Apache Kafka for real-time data streaming, and various cloud services such as Amazon Web Services (AWS) or Google Cloud Platform (GCP) for deployment and data management. Familiarity with tools like Apache Airflow for workflow management and data visualization tools such as Tableau or Looker will also be beneficial.

Ideal Candidates

Successful candidates will be proactive problem solvers with a passion for data and technology. They should possess strong analytical skills and a collaborative mindset to thrive in a remote team environment. A critical thinker who demonstrates adaptability in the face of change will excel in this role. Ideal candidates will also display a commitment to continuous learning, keeping abreast of industry advancements to apply innovative techniques in their work.