Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Smoothstack, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Databricks Engineer at Smoothstack will be responsible for designing, implementing, and managing data solutions using the Databricks platform. This role involves collaborating with data scientists and analysts to optimize data workflows, ensuring seamless integration between various data sources. The engineer will contribute to the development of data pipelines, enhance the performance of data processing jobs, and support the overall data architecture to empower data-driven decision-making across the organization.

Job Responsibilities

• Design and implement scalable data pipelines on the Databricks platform.
• Collaborate with cross-functional teams to gather data requirements and translate them into technical specifications.
• Optimize existing data processes for performance and scalability.
• Monitor and troubleshoot data workflows, ensuring reliability and data integrity.
• Develop and maintain comprehensive documentation for data systems and processes.
• Stay updated with industry trends and best practices in data engineering and Databricks technologies.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of experience in data engineering or a similar role.
• Proficiency in Apache Spark and experience using Databricks for data processing.
• Strong knowledge of SQL and data modeling techniques.
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
• Experience with programming languages such as Python or Scala.
• Excellent problem-solving skills and ability to work collaboratively in a team environment.

Technologies & Tools

This position utilizes a variety of technologies and software tools, including but not limited to Databricks, Apache Spark, SQL databases, cloud services (AWS, Azure, Google Cloud), and programming languages like Python and Scala. Familiarity with version control systems such as Git and data visualization tools is also advantageous.

Ideal Candidates

The ideal candidates for this position possess not only technical expertise but also a proactive approach and strong communication skills. They are detail-oriented, capable of managing multiple projects simultaneously, and have a passion for leveraging data to influence business decisions. A team player attitude and the ability to adapt in a fast-paced environment are crucial for success in this role.