Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Dice, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Databricks Engineer at Dice will be responsible for managing and optimizing data processing and analytics workflows using the Databricks platform. This role involves collaborating with data teams to develop robust data pipelines, ensuring data quality, and promoting best practices in data engineering. The ideal candidate will utilize their expertise in big data technologies and contribute to enabling data-driven decision-making across the organization.

Job Responsibilities

• Design, implement, and maintain data pipelines using Databricks and Apache Spark.
• Collaborate with data scientists and analysts to understand data requirements and provide data solutions.
• Monitor and optimize the performance of data workflows, ensuring high availability and reliability.
• Implement data governance standards and oversee data quality assurance processes.
• Utilize best practices in coding and software engineering to enhance data architecture.
• Manage and configure Azure Databricks environments for optimal performance and cost efficiency.
• Document data processes and build documentation for data engineering best practices.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• 3+ years of experience in data engineering or a similar role.
• Proficiency in Apache Spark and Databricks environments.
• Strong programming skills in Python, Scala, or Java.
• Experience with data modeling, ETL processes, and data warehousing concepts.
• Familiarity with cloud platforms such as Azure or AWS.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

The Databricks Engineer will work extensively with the Databricks platform, leveraging tools such as Apache Spark for big data processing. Additional technologies may include cloud services like Azure, database management systems such as SQL, and data visualization tools to support analytical requirements. Familiarity with version control systems like Git is also beneficial for collaboration among team members.

Ideal Candidates

The ideal candidate for the Databricks Engineer position is detail-oriented, analytical, and possesses a strong desire for continuous improvement. A successful individual in this role will demonstrate the ability to communicate effectively with technical and non-technical stakeholders alike. They should also showcase adaptability to evolving technologies and an eagerness to contribute to a collaborative and innovative workplace culture.