Big Data Software Engineer: Remote

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by iitjobs, Inc., so some details may not match this job. The employer's own page has the real description.

Job Overview

As a Big Data Software Engineer at iitjobs, Inc., you will be tasked with designing and developing scalable big data solutions that process, store, and analyze large datasets. You will play a pivotal role in implementing data pipelines, optimizing performance, and ensuring data integrity. Your expertise in big data technologies will allow you to collaborate effectively with cross-functional teams to drive data-driven decision-making and innovation within the organization.

Job Responsibilities

• Develop and maintain data processing architectures and workflows for efficient data ingestion, ETL (Extract, Transform, Load), and real-time data streaming solutions.
• Collaborate with data scientists and analysts to identify data requirements and deliver high-quality data sets tailored to their needs.
• Optimize performance of data systems, leveraging technologies such as Apache Spark, Hadoop, and other distributed computing frameworks.
• Implement data storage solutions using both SQL and NoSQL databases, ensuring appropriate indexing and data retrieval methodologies.
• Participate in code reviews and troubleshooting sessions to maintain code quality and resolve technical issues.
• Document processes, designs, and updates in a clear and comprehensive manner to facilitate understanding across teams.
• Stay current with evolving big data technologies and methodologies to contribute to continuous improvements in the data ecosystem.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field; Masters preferred.
• Minimum of 3 years’ experience in big data software engineering or a similar role.
• Proficiency in programming languages such as Java, Python, or Scala.
• Strong understanding of distributed computing concepts and architectures.
• Experience with data modeling and database design; familiarity with SQL and NoSQL technologies like MongoDB, Cassandra, or similar.
• Hands-on experience with big data tools such as Apache Hadoop, Spark, Kafka, and other related technologies.
• Excellent problem-solving and analytical skills, with a strong attention to detail.
• Ability to work autonomously and collaboratively within diverse teams.

Technologies & Tools

The Big Data Software Engineer role involves working with various technologies and tools including but not limited to Apache Hadoop, Apache Spark, Apache Kafka, Amazon Web Services (AWS), and Google Cloud Platform (GCP). Familiarity with SQL and NoSQL databases, as well as data visualization tools, will be essential for successfully carrying out the responsibilities associated with this position.

Ideal Candidates

Ideal candidates for this position will possess strong analytical skills, a passion for working with data, and the ability to navigate complex problems efficiently. They will demonstrate excellent communication skills, allowing them to effectively collaborate with both technical and non-technical stakeholders. A proactive attitude and the capacity for self-directed work are crucial to thrive in a remote environment. Candidates should also display a continuous learning mindset, eager to explore new technologies and contribute to innovative data solutions.