Senior Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Steneral Consulting, so some details may not match this job. The employer's own page has the real description.

Job Overview

Steneral Consulting is seeking a highly skilled Senior Databricks Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and implementing robust data solutions using Databricks, as well as optimizing existing workflows to ensure efficiency and scalability. You will collaborate with cross-functional teams to analyze data requirements, provide best practices, and facilitate data-driven decision-making. Your expertise will be instrumental in driving the success of our data initiatives, as well as mentoring junior engineers and promoting a culture of continuous improvement.

Job Responsibilities

• Design and implement data pipelines using Databricks and Apache Spark.
• Optimize and tune data processing workflows for performance and scalability.
• Collaborate with data scientists and analysts to understand data needs and provide innovative solutions.
• Conduct code reviews and provide mentorship to junior engineers within the team.
• Maintain high standards of data quality and integrity throughout the data lifecycle.
• Continuously explore and assess new data technologies and evaluate their applicability to business needs.
• Document architecture, processes, and best practices to facilitate knowledge sharing.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field; a Master’s degree is a plus.
• 5+ years of experience in data engineering, with a strong focus on Databricks and Apache Spark.
• Proficiency in programming languages such as Python, Scala, or Java.
• Extensive experience with SQL and data modeling best practices.
• Strong understanding of data warehousing concepts and experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Excellent problem-solving skills and ability to work in a fast-paced environment.
• Effective communication skills and ability to work collaboratively with cross-functional teams.

Technologies & Tools

The Senior Databricks Engineer will work primarily with Databricks and Apache Spark for data processing and analytics. Additionally, proficiency in cloud services such as AWS, Azure, or Google Cloud is essential, along with tools such as Jupyter Notebooks, SQL, and data visualization software. Familiarity with version control systems like Git and CI/CD pipelines is also required to ensure efficient project management and deployment.

Ideal Candidates

The ideal candidates for this position are driven, analytical thinkers with a passion for working with data. They possess a strong technical background and a willingness to explore innovative solutions to complex challenges. Successful candidates are adaptable team players who thrive in collaborative environments, demonstrating excellent communication skills and leadership capabilities. They are committed to continuous learning and development, both personally and professionally, and are excited about contributing to the success of Steneral Consulting.