Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by LightFeather, so some details may not match this job. The employer's own page has the real description.

Overview

The Databricks Engineer at LightFeather will play a vital role in leveraging data engineering and analytics capabilities within the Databricks environment to drive data-driven insights and streamline processes. This role encompasses designing and implementing data pipelines, optimizing data workflows, and collaborating with cross-functional teams to ensure seamless integration and utilization of data solutions. The ideal candidate will be responsible for enhancing data quality, scalability, and security, while also contributing to the continuous improvement of data practices across the organization.

Job Responsibilities

• Develop, enhance, and maintain robust data pipelines and workflows in the Databricks platform.
• Collaborate with data scientists and business analysts to understand data requirements and translate them into technical workflows.
• Ensure data integrity, security, and governance in accordance with company policies.
• Optimize data storage and processing to maximize performance and reduce costs.
• Implement automated testing and monitoring to ensure the reliability and accuracy of data processes.
• Participate in data model design and architecture discussions to support evolving business needs.
• Provide guidance and support to team members in best practices for data engineering and analytics.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of experience in data engineering or a similar role.
• Proficiency in Apache Spark and Databricks platform functionalities.
• Solid understanding of ETL processes, data warehousing, and data modeling techniques.
• Experience with SQL and programming languages such as Python or Scala.
• Familiarity with cloud platforms (e.g., AWS, Azure) and their data services.
• Strong problem-solving and analytical skills with attention to detail.

Technologies & Tools

The Databricks Engineer will work extensively with the Databricks platform, leveraging its data processing capabilities to develop scalable solutions. Other tools and technologies relevant to this position include Apache Spark, SQL, Python, and cloud services provided by platforms such as AWS or Azure. Familiarity with data visualization tools such as Tableau or Power BI is also advantageous in this role.

Ideal Candidates

The ideal candidate for this position will possess a strong technical aptitude, combined with excellent communication and collaboration skills. They should be proactive, detail-oriented, and capable of managing multiple tasks effectively in a fast-paced environment. A passion for data and a continuous improvement mindset will also contribute greatly to success in this role, as the ideal candidate should be eager to learn and adapt to emerging technologies and best practices in data engineering.