DataBricks Lead Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Bahwan CyberTek, so some details may not match this job. The employer's own page has the real description.

Job Overview

As a DataBricks Lead Engineer at Bahwan CyberTek, you will play a pivotal role in leading and managing data engineering projects leveraging the DataBricks platform. You will be responsible for designing and implementing data solutions that drive analytics, reporting, and operational efficiencies. The ideal candidate will collaborate with cross-functional teams to architect and maintain scalable and high-performance data pipelines, ensuring data quality and availability for diverse business applications.

Job Responsibilities

• Lead the design and development of data pipelines using DataBricks and Apache Spark.
• Collaborate with stakeholders to define data requirements and implement data ingestion processes.
• Ensure the reliability, performance, and scalability of data solutions.
• Mentor and guide junior engineers in best practices and methodologies in data engineering.
• Conduct performance tuning and optimization of data processing jobs.
• Implement data governance and data quality standards.
• Support production issues by troubleshooting and debugging data-related incidents.
• Stay updated on emerging technologies and industry trends related to data engineering and analytics.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 5+ years of experience in data engineering, with a focus on DataBricks or similar platforms.
• Strong proficiency in Apache Spark, SQL, and ETL processes.
• Experience with cloud services such as AWS, Azure, or GCP.
• Understanding of data warehousing concepts and big data technologies.
• Familiarity with programming languages such as Python, Scala, or Java.
• Excellent analytical skills and problem-solving capabilities.
• Strong communication skills, with the ability to work collaboratively across teams.

Technologies & Tools

The position requires proficiency in several technologies and tools, including DataBricks, Apache Spark, SQL, cloud platforms (AWS, Azure, or GCP), and data visualization tools such as Tableau or Power BI. Familiarity with programming languages like Python, Scala, or Java is also important, as well as knowledge of CI/CD pipelines and version control systems like Git.

Ideal Candidates

The ideal candidates for the DataBricks Lead Engineer position will possess a robust technical background, complemented by strong leadership qualities. They will be detail-oriented, able to navigate complex data challenges with ease, and skilled in fostering collaboration among team members. A proactive mindset, with a passion for learning and innovation, will help in driving data initiatives that align with business strategies and contribute to organizational success.