Software Engineer, Data Pipelines

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by EvenUp, so some details may not match this job. The employer's own page has the real description.

Job Overview

As a Software Engineer specializing in Data Pipelines at EvenUp, you will play a pivotal role in designing, developing, and maintaining robust data pipelines that enable efficient and scalable data processing. You will collaborate closely with cross-functional teams to gather requirements, implement data solutions, and improve current workflows. Your expertise in software engineering principles and data architecture will be critical in ensuring data accessibility, reliability, and performance, which are essential for our data-driven decisions.

Job Responsibilities

• Design and develop efficient data pipelines for data ingestion, transformation, and storage.
• Collaborate with data scientists and analysts to understand data requirements and create scalable solutions.
• Implement monitoring and optimization strategies to ensure data pipeline reliability and performance.
• Maintain and enhance existing data processing frameworks to meet evolving business needs.
• Conduct thorough testing to validate data accuracy and processing logic.
• Document processes, architecture, and any relevant technical specifications for future reference.
• Participate in code reviews and foster best practices in software development.
• Stay updated with new technologies and methodologies in data engineering and propose improvements.

Qualifications

• Bachelor’s degree in Computer Science, Software Engineering, or a related field.
• Minimum of 3 years of experience in software engineering or data pipeline development.
• Proficiency in Python and experience with frameworks like Apache Airflow or Apache Spark.
• Strong understanding of SQL and experience with relational databases such as PostgreSQL or MySQL.
• Familiarity with cloud services such as AWS, Google Cloud, or Azure.
• Experience with ETL processes and familiarity with data warehousing solutions.
• Strong analytical skills and attention to detail.
• Excellent problem-solving abilities and capacity to work collaboratively in teams.

Technologies & Tools

The position of Software Engineer, Data Pipelines at EvenUp will utilize a variety of technologies and tools including Python for scripting, Apache Airflow for workflow management, SQL for database querying, and Apache Spark for big data processing. Familiarity with cloud platforms, such as AWS and Google Cloud, will also be essential, particularly for implementing scalable solutions. Additionally, knowledge of version control systems like Git will be important for collaborative development and code management.

Ideal Candidates

The ideal candidates for the Software Engineer, Data Pipelines position at EvenUp will exhibit a strong technical foundation, demonstrating proficiency in data engineering principles and practices. They will be detail-oriented, exhibiting an ability to focus on accuracy and quality in their work. Excellent communication skills are vital, as collaboration across teams is essential. Furthermore, the right candidate will possess a passion for problem-solving, a proactive attitude toward continuing education, and an enthusiastic approach to embracing new technologies and methodologies in the ever-evolving field of data engineering.