Software Engineer, Data Pipelines

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by EvenUp, so some details may not match this job. The employer's own page has the real description.

Job Overview

As a Software Engineer focusing on Data Pipelines at EvenUp, you will be responsible for designing, implementing, and optimizing data workflow solutions that facilitate data ingestion, processing, and storage within our dynamic environment. Your work will significantly impact data accessibility and utilization by development teams and end-users. You will collaborate with cross-functional teams to ensure that data infrastructure supports analytical and operational requirements efficiently, adhering to best practices in coding, testing, and deployment.

Job Responsibilities

• Design and develop robust data pipelines to automate data extraction, transformation, and loading processes.
• Optimize existing data flows for performance, scalability, and reliability.
• Collaborate with data scientists and analysts to understand data requirements and deliver data solutions accordingly.
• Monitor and troubleshoot data pipeline failures or issues in a timely manner, ensuring data integrity and availability.
• Implement best practices for version control, testing, and deployment of data pipeline code.
• Document data workflows and architectures to facilitate knowledge sharing within the technical team.
• Stay current with industry trends and technologies to continually improve data pipeline frameworks and processes.

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, or a relevant field.
• 3+ years of experience in software development, specifically in data pipeline or ETL (Extract, Transform, Load) processes.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with big data technologies such as Apache Hadoop, Apache Spark, or Apache Kafka.
• Familiarity with data warehousing solutions (e.g., AWS Redshift, Google BigQuery).
• Strong understanding of SQL and databases, including relational and NoSQL databases.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

The ideal candidate will have experience with a mix of cloud and on-premise technologies, including AWS, Azure, or GCP, and data processing frameworks such as Apache Airflow, Talend, or similar. Knowledge of containerization tools like Docker and Kubernetes is a plus, as well as familiarity with version control systems such as Git. Proficiency in using monitoring tools (e.g. Grafana, Prometheus) to ensure the performance and reliability of data pipelines is also desirable.

Ideal Candidates

The ideal candidate is a proactive and analytical thinker who thrives in fast-paced environments. You are detail-oriented with a strong ability to troubleshoot and resolve technical challenges quickly. Your effective communication skills enable you to collaborate seamlessly with team members across different functions. You exhibit a passion for technology and innovation, continuously seeking to enhance your skills and contribute to the success of the organization.