Senior Software Engineer (Data Pipelines)

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Wpromote, so some details may not match this job. The employer's own page has the real description.

Job Overview

Wpromote is seeking a highly skilled Senior Software Engineer specializing in Data Pipelines to join our dynamic team. In this role, you will be responsible for designing, developing, and optimizing data pipeline solutions that facilitate the ingestion, processing, and storage of vast amounts of data. You will work closely with cross-functional teams to ensure data integrity and performance while driving forward our mission to leverage data for enhanced marketing strategies. Your expertise will be pivotal in creating scalable and reliable infrastructure that meets the growing demands of our clients.

Job Responsibilities

• Design and implement robust data pipeline architectures to ingest, process, and store large datasets.
• Collaborate with data scientists and analysts to understand data requirements and optimize data workflows.
• Monitor and maintain data pipeline performance, ensuring efficient data flow and minimizing downtime.
• Develop automated data validation and monitoring processes to ensure data integrity and quality.
• Conduct code reviews and mentor junior engineers to foster a culture of continuous improvement.
• Stay updated with industry best practices and emerging technologies related to data engineering.
• Document data pipeline designs, processes, and workflows for reference and onboarding purposes.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field; a Master’s degree is a plus.
• 5+ years of experience in software engineering, with a strong focus on data pipelines and ETL processes.
• Proficiency in programming languages such as Python, Java, or Scala.
• Hands-on experience with data warehousing solutions like AWS Redshift, Google BigQuery, or Snowflake.
• Familiarity with big data technologies including Hadoop, Spark, or Kafka.
• Strong understanding of SQL and NoSQL databases.
• Experience with cloud platforms such as AWS, GCP, or Azure is highly desirable.
• Excellent problem-solving skills and ability to work in fast-paced environments.

Technologies & Tools

The Senior Software Engineer (Data Pipelines) at Wpromote will utilize a variety of technologies and tools to create efficient data processes. Key technologies include Apache Spark for big data processing, AWS and Google Cloud for cloud-based solutions, and Apache Kafka for real-time data streaming. Additional tools may include SQL databases for data querying, Tableau for data visualization, and Git for version control. Familiarity with containers and orchestration frameworks such as Docker and Kubernetes will also be beneficial.

Ideal Candidates

Ideal candidates for this position will possess a strong analytical mindset and a passion for leveraging data to drive business results. They should demonstrate exceptional communication skills, enabling them to collaborate effectively across teams. A proactive approach to problem-solving and a robust understanding of best engineering practices are crucial. Moreover, candidates should be adaptable in a fast-evolving technology landscape, eager to learn and implement new solutions that enhance operational efficiencies.