Databricks Platform Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Dice, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Databricks Platform Engineer at Dice will play a crucial role in designing, implementing, and maintaining robust data processing architectures on the Databricks platform. The successful candidate will collaborate closely with data engineering teams to enhance data workflows, optimize performance, and ensure the reliability of data systems. The role requires a strong focus on automation, scalability, and data security while leveraging advanced analytics to drive business insights. Additionally, the Databricks Platform Engineer will be responsible for troubleshooting issues and recommending best practices for operational excellence.

Job Responsibilities

• Design and implement scalable data architectures on the Databricks platform.
• Collaborate with data teams to develop, test, and deploy robust data pipelines.
• Monitor, troubleshoot, and optimize the performance of data workflows.
• Establish and maintain best practices for data governance and security.
• Create and manage documentation for all data processes and systems.
• Automate routine operational tasks and improve system reliability.
• Participate in architecture discussions and provide insights on technology trends.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Proven experience working with Databricks and Apache Spark.
• Strong proficiency in programming languages such as Python, Scala, or SQL.
• Experience with cloud services such as AWS, Azure, or GCP.
• Knowledge of ETL processes and data warehousing concepts.
• Familiarity with data visualization tools and methodologies.
• Excellent analytical skills and attention to detail.

Technologies & Tools

The Databricks Platform Engineer will utilize a variety of technologies and tools to perform their job effectively. Key technologies include Databricks for data orchestration, Apache Spark for data processing, and cloud platforms like AWS, Azure, or Google Cloud Platform for hosted services. Additionally, skills in SQL, Python, or Scala will be essential for developing and optimizing data pipelines, while familiarity with data visualization tools such as Power BI or Tableau will enhance reporting capabilities.

Ideal Candidates

The ideal candidates for the Databricks Platform Engineer position are individuals with a strong technical background who exhibit an aptitude for problem-solving and innovation. They should possess excellent communication skills, enabling them to collaborate effectively with cross-functional teams. A proactive mindset and a passion for data engineering and analytics are essential, along with a dedication to continuous learning and improvement in the ever-evolving data landscape. Candidates should also demonstrate a commitment to best practices in data management and security.