Data Engineering Manager(Databricks exp)

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Diligente Technologies, so some details may not match this job. The employer's own page has the real description.

Job Overview

Diligente Technologies is seeking a highly skilled Data Engineering Manager with extensive experience in Databricks to lead and develop our data team. The successful candidate will be responsible for overseeing the design, implementation, and maintenance of robust data pipelines while ensuring data accuracy, reliability, and accessibility. This role requires exceptional leadership capabilities, a deep understanding of data engineering practices, and a strategic mindset to drive data-driven decision-making across the organization.

Job Responsibilities

• Lead the data engineering team to design, implement, and maintain scalable data infrastructure utilizing Databricks.
• Oversee the development of ETL processes to ensure data quality and integrity across various sources.
• Collaborate with business stakeholders to understand data needs and define project scope and objectives.
• Monitor and optimize data pipeline performance to enhance efficiency and reduce latency.
• Implement best practices in data governance, security, and compliance.
• Mentor and train junior data engineers to enhance their skills and professional growth.
• Stay current with emerging trends and technologies in data engineering and analytics.

Qualifications

• Bachelor's degree in Computer Science, Data Science, or a related field; a Master's degree is a plus.
• Minimum of 5 years of experience in data engineering, with at least 2 years in a managerial role.
• Proven experience with Databricks and associated data tools.
• Strong programming skills in languages such as Python, Scala, or Java.
• Deep understanding of cloud platforms like AWS, Azure, or Google Cloud.
• Familiarity with data warehousing concepts and SQL technologies.
• Excellent analytical and problem-solving skills, with a high attention to detail.
• Strong communication and interpersonal skills to interact effectively with stakeholders at all levels.

Technologies & Tools

The Data Engineering Manager will engage with a variety of technologies and tools such as Databricks for data processing and analytics, Apache Spark for large-scale data processing, SQL databases for data storage, and cloud services like AWS or Azure for scalable infrastructure. Familiarity with tools such as Apache Kafka, Airflow, and various ETL frameworks will be increasingly beneficial in maximizing project outcomes.

Ideal Candidates

The ideal candidate for this position will be a forward-thinking leader with a passion for data and technology. They will possess a strong analytical mindset, able to navigate complex data challenges and translate them into actionable solutions. The candidate should demonstrate a collaborative spirit, proactively engage with cross-functional teams, and foster an inclusive environment for innovation. Strong organizational skills and the ability to prioritize tasks in a fast-paced setting are essential to meet business objectives effectively.