Databricks Platform Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Planet Technology, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Databricks Platform Engineer at Planet Technology will play a crucial role in the design, implementation, and maintenance of the Databricks platform to support our data analytics and engineering teams. This position involves collaborating with cross-functional teams to ensure optimal performance and scalability of data solutions while also contributing to architecture and best practices for data management. The candidate will be responsible for performance tuning, troubleshooting, and implementing automation processes within the Databricks environment, ensuring data accessibility and reliability for various stakeholders.

Job Responsibilities

• Design and implement scalable and efficient data pipelines using Databricks.
• Optimize and troubleshoot existing data workflows to improve performance and reliability.
• Collaborate with data engineers and data scientists to ensure data solutions meet organizational needs.
• Monitor system performance and provide solutions for issues as they arise.
• Implement and manage security protocols and governance measures for data access.
• Develop and maintain documentation related to architecture, processes, and standards.
• Automate repetitive tasks to enhance efficiency within the Databricks platform.
• Stay updated on industry trends and best practices related to Databricks and big data technologies.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 3+ years of experience in a data engineering or platform engineering role.
• Proficiency in Databricks and Spark, with hands-on experience in data processing.
• Strong programming skills in languages such as Python, Scala, or Java.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Familiarity with ETL processes and data warehousing concepts.
• Knowledge of data modeling, database design, and data governance best practices.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

The Databricks Platform Engineer will work with several cutting-edge technologies and tools, including Databricks for data analytics and machine learning, Apache Spark for large-scale data processing, cloud services such as AWS or Azure for hosting, and version control systems like Git for code management. Additionally, familiarity with tools like Jupyter Notebooks, Airflow for workflow management, and CI/CD pipelines for deployment automation will be advantageous.

Ideal Candidates

The ideal candidate for the Databricks Platform Engineer position will exhibit a deep passion for data technology and a strong desire to innovate within the platform. They will possess a combination of analytical skills and technical prowess, making them adept at solving complex problems and optimizing data workflows. Effective collaboration and communication skills are essential, as this role requires working closely with various teams. A proactive and self-motivated attitude, along with a commitment to continuous learning and improvement, will elevate a candidate's potential for success at Planet Technology.