Senior Databricks Engineer (Remote)

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Dice, so some details may not match this job. The employer's own page has the real description.

Job Overview

We are seeking a highly skilled Senior Databricks Engineer to join our remote team at Dice. In this role, you will be responsible for designing and implementing robust data pipelines, optimizing analytics processes, and collaborating with cross-functional teams to drive data-driven decision-making. Your expertise in Databricks will be crucial to enhance our data architecture, ensuring efficient data management and enabling powerful analytics solutions that support various business objectives.

Job Responsibilities

• Design, develop, and maintain scalable data processing frameworks using Databricks and Apache Spark.
• Collaborate with data scientists and analysts to identify and prioritize data requirements and analytics processes.
• Implement best practices for data ingestion, transformation, and storage within a cloud environment.
• Optimize and tune Spark jobs for performance and cost savings while monitoring job outcomes.
• Conduct quality assurance on data pipelines to ensure accuracy and reliability of data outputs.
• Document processes, architectures, and data models to support team knowledge sharing and onboarding.
• Stay updated with emerging technologies and methodologies in data engineering and analytics.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field; Master’s degree preferred.
• Minimum of 5 years of experience in data engineering or a related field.
• Proficiency in Databricks, Apache Spark, and big data technologies.
• Strong programming skills in Python, Scala, or Java.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Familiarity with data warehousing concepts and technologies such as Snowflake or Redshift.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

This position requires proficiency with various technologies and tools essential for data processing and analytics. Key technologies include Databricks for data engineering, Apache Spark for distributed computing, and cloud services such as AWS or Azure for data storage and browser management. Familiarity with data visualization tools and ETL frameworks will also be beneficial in enhancing our analytics capabilities.

Ideal Candidates

The ideal candidates for the Senior Databricks Engineer position will have a strong analytical mindset, with a passion for data-driven methodologies. They should exhibit excellent collaboration and communication skills, essential for working effectively with diverse teams. An ability to think critically and adapt to evolving technologies will enable them to contribute to innovative data solutions that align with the strategic goals of the organization.