Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by LightFeather, so some details may not match this job. The employer's own page has the real description.

Job Overview

LightFeather is seeking a skilled Databricks Engineer to join our dynamic team. In this role, you will be responsible for designing, implementing, and optimizing data processing solutions using Databricks and associated technologies. Your expertise will directly contribute to the development of scalable data pipelines and analytics solutions that drive business insights. The ideal candidate will collaborate closely with data scientists, data engineers, and other stakeholders to ensure effective data operations while maintaining high standards of performance and reliability.

Job Responsibilities

• Design and develop scalable data processing solutions on the Databricks platform.
• Optimize data workflows and improve ETL processes to enhance performance and reliability.
• Collaborate with data scientists and analysts to support data-driven initiatives and analytics projects.
• Implement best practices for data governance, security, and compliance within the Databricks environment.
• Monitor and troubleshoot data pipeline issues to ensure minimal downtime and efficient data flows.
• Provide technical guidance and mentorship to junior engineers and team members.
• Document data pipelines, processes, and solutions for future reference and knowledge sharing.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• Minimum of 3 years of experience in data engineering or a similar role, specifically with Databricks.
• Proficiency in Apache Spark, SQL, and data processing frameworks.
• Strong programming skills in Python or Scala.
• Experience with cloud platforms, particularly Azure or AWS.
• Familiarity with data modeling, data warehousing, and ETL best practices.
• Knowledge of data governance and compliance standards.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

The Databricks Engineer role at LightFeather will involve the use of various technologies and tools, including but not limited to Databricks, Apache Spark, SQL, Python, Azure or AWS cloud services, and data orchestration tools like Apache Airflow. Familiarity with version control systems, such as Git, is also beneficial.

Ideal Candidates

The ideal candidate for the Databricks Engineer position will possess a strong analytical mindset combined with exceptional technical skills. Candidates should demonstrate a proactive approach to problem-solving and a collaborative spirit, as teamwork is crucial in our fast-paced environment. Strong communication skills and the ability to articulate complex technical concepts to non-technical stakeholders are essential. Additionally, a genuine passion for data and emerging technologies will further enhance an individual's fit for this role.