Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by LightFeather, so some details may not match this job. The employer's own page has the real description.

Job Overview

LightFeather is seeking a highly skilled Databricks Engineer to join our dynamic team. The successful candidate will be responsible for designing, developing, and deploying robust data solutions on the Databricks platform. This role will involve collaborating with data scientists, analysts, and system architects to design data pipelines, optimize performance, and ensure data integrity across various projects. With a focus on innovation and efficiency, the Databricks Engineer will play a critical role in shaping our data strategy and enabling data-driven decision-making within the organization.

Job Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
• Collaborate with cross-functional teams to gather requirements and develop data solutions that meet business needs.
• Implement data quality checks and optimize data processing workflows for performance.
• Monitor and troubleshoot ETL processes, ensuring data integrity and availability.
• Support data modeling efforts to enable effective data analytics and reporting.
• Document processes, architectures, and data flows to ensure clarity and compliance.
• Stay updated on emerging technologies and industry trends to enhance our data strategies.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• 5+ years of hands-on experience in data engineering and working with data transformation tools.
• Proficiency in Databricks platform and Apache Spark.
• Strong knowledge of SQL and experience with relational databases.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Familiarity with data warehousing concepts and big data technologies.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

The Databricks Engineer will work extensively with the Databricks platform, utilizing Apache Spark for data processing and analytics. Proficiency in SQL is essential for querying databases, while familiarity with cloud services such as AWS, Azure, or Google Cloud Platform is also critical. The role may require leveraging data visualization tools and frameworks that integrate seamlessly with the Databricks environment to support data-driven insights.

Ideal Candidates

Ideal candidates for the Databricks Engineer position at LightFeather will possess strong analytical skills and a passion for data-driven solutions. They will be effective communicators, able to translate technical concepts to non-technical stakeholders, and demonstrate a collaborative mindset to work alongside various teams. A proactive approach to identifying problems and implementing efficient solutions is essential, alongside a commitment to continuous learning and adapting to new technologies. Enthusiasm for innovation and an eagerness to contribute to team success will set the ideal candidate apart.