Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by LightFeather, so some details may not match this job. The employer's own page has the real description.

Overview

The Databricks Engineer at LightFeather will play a crucial role in enhancing our data processing capabilities by leveraging the Databricks platform for large-scale data analytics and engineering projects. The ideal candidate will be responsible for designing, building, and optimizing data workflows while collaborating closely with data scientists and analysts to deliver high-quality data solutions that drive business insights. This position demands strong technical skills, creativity, and the ability to adapt in a fast-paced environment, ensuring data integrity and security throughout all processes.

Job Responsibilities

• Design and implement scalable data pipelines and ETL processes using Databricks.
• Optimize data processing workflows and enhance performance metrics.
• Collaborate with cross-functional teams to gather requirements and translate them into technical specifications.
• Support data ingestion, transformation, and storage within the cloud environment.
• Monitor and troubleshoot data workflows and resolve data quality issues promptly.
• Implement security best practices to protect sensitive data.
• Document processes, architectures, and solutions using best industry practices.
• Stay updated on Databricks features, enhancements, and industry trends.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in data engineering, data analysis, or related roles.
• Strong proficiency in Databricks, Apache Spark, and related data processing technologies.
• Experience with cloud platforms, preferably AWS or Azure.
• Solid understanding of SQL and experience in data modeling.
• Familiarity with data visualization tools such as Tableau or Power BI.
• Ability to write clean, maintainable code in Python, Scala, or Java.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

The role of a Databricks Engineer at LightFeather requires proficiency in a variety of modern technologies and tools. The primary platform used will be Databricks, as well as Apache Spark for data processing tasks. Familiarity with cloud services such as AWS or Azure, alongside SQL databases, will be essential. Additional tools may include Python for scripting, Git for version control, and data visualization software like Tableau or Power BI for reporting purposes.

Ideal Candidates

The ideal candidate for the Databricks Engineer role will possess a strong analytical mindset paired with a collaborative spirit. They will be self-motivated and able to manage multiple projects simultaneously while adhering to deadlines. A proactive learner who keeps abreast of the latest advancements in data engineering and analytics, the candidate should also demonstrate effective communication skills and the ability to work effectively within a team-oriented environment. Strong organizational skills and a commitment to delivering high-quality results will distinguish the ideal candidate for this position.