Lead Data Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Optomi, so some details may not match this job. The employer's own page has the real description.

Overview

Optomi is seeking a highly skilled and motivated Lead Data Engineer to join our dynamic technology team. The Lead Data Engineer will be responsible for designing, implementing, and maintaining scalable data processing systems, ensuring that our data architecture is optimized for processing large volumes of data efficiently. This individual will lead a team of data engineers in leveraging modern data engineering practices and tools, driving data-driven decision-making across the organization. The ideal candidate will possess strong leadership skills and a deep understanding of data technologies.

Job Responsibilities

• Design and develop robust, scalable, and secure data pipelines to automate data ingestion and processing.
• Lead and mentor a team of data engineers, fostering a culture of innovation and continuous improvement.
• Collaborate with data scientists, analysts, and stakeholders to understand data requirements and provide solutions that meet business needs.
• Implement best practices for data governance, data quality, and data lifecycle management.
• Monitor and optimize system performance, reliability, and data accessibility.
• Conduct code reviews to ensure adherence to coding standards and best practices.
• Stay updated with the latest developments in data engineering tools and technologies, and introduce them to the team as appropriate.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field; Master's degree preferred.
• 5+ years of experience in data engineering or data architecture.
• Strong proficiency in programming languages such as Python, Java, or Scala.
• Extensive experience with data warehousing solutions like Amazon Redshift, Google BigQuery, or Snowflake.
• Proficient in ETL tools such as Apache Airflow, Talend, or Informatica.
• Familiarity with cloud platforms (AWS, Azure, Google Cloud) and their data services.
• Knowledge of big data technologies such as Hadoop, Spark, or Kafka.
• Excellent analytical, problem-solving and communication skills.

Technologies & Tools

The Lead Data Engineer will work with a variety of technologies and tools to support data architecture and engineering efforts. The role will involve utilizing cloud-based platforms such as AWS or Azure for data storage and processing, employing ETL tools like Apache Airflow and Talend for data integration, and leveraging big data frameworks such as Apache Spark and Hadoop for large-scale data processing. Familiarity with SQL for data querying, as well as experience in containerization tools like Docker or Kubernetes, will also be beneficial.

Ideal Candidates

The ideal candidate for the Lead Data Engineer position at Optomi will demonstrate strong leadership capabilities, excellent communication skills, and a collaborative spirit. They will be proactive problem solvers who are adaptable to change and capable of managing multiple projects simultaneously. A genuine passion for technology and a desire to innovate within the data engineering field are essential. Candidates should be committed to best practices in data management and maintaining high data quality standards, and possess a keen ability to translate complex technical concepts into accessible language for both technical and non-technical stakeholders.