Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Flexton Inc., so some details may not match this job. The employer's own page has the real description.

Job Overview

Flexton Inc. is seeking a skilled Databricks Engineer who will be responsible for designing, implementing, and maintaining data pipelines on the Databricks platform. The successful candidate will work closely with data scientists and analysts to ensure that data is accessible, reliable, and efficiently processed. This role involves utilizing advanced analytics and big data technologies to solve complex data challenges, thereby empowering the organization with actionable insights. The Databricks Engineer will contribute to architectural design decisions, build robust data solutions, and optimize existing processes to enhance performance and scalability.

Job Responsibilities

• Design, develop, and maintain data pipelines using Databricks and Apache Spark.
• Collaborate with data scientists and analysts to understand data requirements and deliver scalable solutions.
• Monitor data quality and performance, troubleshoot issues, and implement solutions.
• Optimize data ingestion and storage strategies to ensure efficiency and cost-effectiveness.
• Document processes, data flows, and architecture to provide clarity and support knowledge sharing.
• Implement best practices in data governance, ensuring compliance with data policies.
• Stay abreast of the latest developments in Databricks and other relevant technologies.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• 5+ years of experience in data engineering or big data solutions with a focus on Databricks.
• Proficient in Apache Spark, SQL, and Python or Scala programming languages.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Strong understanding of data modeling, ETL processes, and data warehousing concepts.
• Experience with version control systems, particularly Git.
• Excellent problem-solving skills and ability to work under tight deadlines.

Technologies & Tools

The Databricks Engineer will work with a variety of technologies including Apache Spark, Databricks, Apache Kafka, AWS/Azure/GCP cloud services, SQL databases, and various ETL tools. Familiarity with data visualization software such as Tableau or Power BI may also be beneficial for collaborating with analytics teams.

Ideal Candidates

The ideal candidate for this position will possess strong analytical and critical thinking skills, with a proactive approach to problem-solving. They should demonstrate a collaborative mindset, ready to work alongside cross-functional teams in a dynamic environment. Effective communication skills are essential for translating complex technical concepts to non-technical stakeholders. The candidate should be self-motivated, adaptable, and eager to learn new technologies and methodologies in the rapidly evolving data landscape.