Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by SysMind, so some details may not match this job. The employer's own page has the real description.

Overview

The Databricks Engineer at SysMind will play a pivotal role in designing, implementing, and maintaining scalable analytics solutions that leverage the Databricks platform. This position entails collaborating closely with data scientists, analysts, and other stakeholders to ensure data pipelines are reliable, efficient, and meet the organization's analytical needs. The ideal candidate will be responsible for optimizing performance and ensuring system security while promoting best practices in data governance and architecture.

Job Responsibilities

• Design, build, and maintain robust data pipelines and ETL processes on the Databricks platform.
• Collaborate with data scientists and analysts to optimize data processing workflows and analytical models.
• Implement data governance policies to ensure data quality, security, and compliance.
• Monitor and troubleshoot data pipeline performance and optimize for scalability and reliability.
• Develop and maintain documentation for data processes, architectures, and workflows.
• Stay abreast of industry trends and emerging technologies to continually enhance the company’s data solutions.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Data Science, or related field.
• 3+ years of experience in data engineering or related roles, specifically with Databricks or Apache Spark.
• Strong programming skills in languages such as Python, Scala, or Java.
• Proficiency in data modeling, ETL processes, and data warehousing concepts.
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
• Experience with SQL and data querying languages.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.

Technologies & Tools

The Databricks Engineer will work with a variety of technologies and tools including Databricks, Apache Spark, SQL, Python, and cloud services such as AWS or Azure. Familiarity with data visualization tools and frameworks like Tableau or Power BI, as well as source control systems like Git, is also beneficial for the role. The engineer will implement and use these tools to streamline analytics processes and enhance data accessibility across teams.

Ideal Candidates

Ideal candidates for the Databricks Engineer position possess a strong technical foundation combined with innovative problem-solving skills. They are proactive and detail-oriented, demonstrating a willingness to embrace new challenges. Effective communication skills and a collaborative spirit to work with various stakeholders are essential qualities. Additionally, a continuous learning mindset and a passion for data-driven insights are characteristics that will lead to success in this role.