Sr. Data Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Vaco, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Sr. Data Engineer at Vaco will be responsible for designing, constructing, installing, and maintaining large-scale processing systems and data pipelines to facilitate data driven decisions. This role requires collaborating with both data scientists and data analytics teams to ensure the architecture is set in a way that optimizes data retrieval systems, supports various data applications, and aligns with overall organizational goals. The ideal candidate will play a pivotal role in addressing complex data challenges while ensuring the integrity, accessibility, and security of data assets within the company.

Job Responsibilities

• Design and implement robust data processing systems and architecture for high-volume datasets.
• Collaborate with data scientists and analysts to understand data requirements and translate them into efficient data models.
• Lead data pipeline development while integrating various data sources and ensuring optimal data flows.
• Optimize and maintain existing data warehousing solutions and ETL processes for enhanced performance.
• Implement data quality controls and automated monitoring systems to ensure data integrity.
• Document data architecture and data pipelines, ensuring compliance with best practices and data governance policies.
• Stay abreast of industry trends and emerging technologies to enhance data engineering capabilities.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field; Master’s degree is a plus.
• Minimum of 5 years of experience in data engineering, data analytics, or a related field.
• Proficient in programming languages such as Python, Java, or Scala.
• Extensive experience with SQL and NoSQL databases such as PostgreSQL, MongoDB, or similar technologies.
• Familiarity with ETL tools and frameworks like Apache NiFi, Talend, or similar.
• Knowledge of cloud platforms such as AWS, Azure or Google Cloud Platform.
• Ability to troubleshoot and optimize performance for large datasets and transaction processing systems.

Technologies & Tools

The Sr. Data Engineer will work with an array of technologies including but not limited to Apache Spark for data processing, cloud services such as Amazon S3 for data storage, and BI tools such as Tableau or Power BI for data visualization. Familiarity with containerization tools like Docker and orchestration platforms like Kubernetes will also be advantageous, as well as experience with Git for version control and collaboration on code repositories.

Ideal Candidates

Ideal candidates for the Sr. Data Engineer role will possess strong analytical and problem-solving skills that enable them to handle complex data tasks proficiently. They should demonstrate effective communication skills to collaborate with cross-functional teams, a proactive approach to meeting deadlines, and a relentless curiosity to explore and implement innovative data solutions. Furthermore, candidates should exhibit a commitment to continual learning, leveraging both peer knowledge and ongoing education to enhance their technical proficiency in an ever-changing data landscape.