Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by LightFeather, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Databricks Engineer at LightFeather will play a pivotal role in leveraging our data analytics capabilities to optimize business processes and drive data-driven decision-making. The successful candidate will be responsible for designing, implementing, and maintaining data pipelines and workflows on the Databricks platform. You will collaborate closely with data scientists, analysts, and other stakeholders to ensure that our data environment is robust, efficient, and scalable, ultimately contributing to our organization's success in harnessing big data analytics.

Job Responsibilities

• Design, develop, and optimize ETL processes and data pipelines using Apache Spark and Databricks.
• Collaborate with cross-functional teams to gather business requirements and translate them into effective data solutions.
• Implement best practices for data governance, security, and quality assurance within the Databricks environment.
• Monitor and troubleshoot data workflows, ensuring high availability and performance for data processing tasks.
• Participate in architecture discussions to enhance our data platform and drive continuous improvement.
• Provide technical guidance and mentorship to junior engineers and data team members.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Data Science, or a related field.
• Minimum of 3 years of experience with Databricks, Apache Spark, or other big data technologies.
• Strong programming skills in languages such as Python, Scala, or Java.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Proficiency in SQL and working with relational databases.
• Familiarity with data modeling, data warehousing concepts, and data analytics tools.

Technologies & Tools

The Databricks Engineer will work extensively with Apache Spark and Databricks for data engineering tasks. Familiarity with cloud services, particularly AWS or Azure, is essential, as well as tools like Apache Kafka for streaming data, and BI tools like Tableau or Power BI for data visualization. You will also employ version control systems such as Git and software development best practices to maintain code quality and project documentation.

Ideal Candidates

Ideal candidates for the Databricks Engineer position possess a strong analytical mindset and a detail-oriented approach to problem-solving. Excellent communication skills are essential, as the role requires collaboration with various teams to understand their data needs and provide insightful solutions. A proactive attitude towards continuous learning and improvement will help candidates thrive in the dynamic environment at LightFeather. The ability to work efficiently in a fast-paced setting while managing multiple priorities will also be key to success in this role.