Databricks Platform/DevOps Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by NLB Services, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Databricks Platform/DevOps Engineer at NLB Services plays a pivotal role in the development, deployment, and optimization of data engineering pipelines and DevOps processes within the Databricks environment. This position involves collaborating with cross-functional teams to ensure robust data processing, automation of workflows, and continuous integration and delivery of high-quality software solutions. The ideal candidate will contribute to building a scalable and efficient data infrastructure, ensuring the reliability and performance of data-centric applications while adhering to best practices in data management and DevOps methodologies.

Job Responsibilities

• Design, implement, and maintain scalable data pipelines utilizing the Databricks platform.
• Automate deployment and monitoring processes within the DevOps framework.
• Collaborate with data scientists and analysts to define data requirements and architecture.
• Implement best practices for data governance, data quality, and security measures.
• Monitor system performance and optimize resource utilization for cost-effective operations.
• Support troubleshooting and debugging of data processing workflows and applications.
• Contribute to the continuous improvement of DevOps processes and methodologies.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.
• 3+ years of experience in Data Engineering, DevOps, or related roles.
• Proficiency in Apache Spark and the Databricks platform.
• Strong knowledge of cloud platforms, preferably AWS, Azure, or Google Cloud.
• Experience with CI/CD tools such as Jenkins, GitLab CI, or similar.
• Familiarity with containerization technologies like Docker and Orchestration tools like Kubernetes.
• Solid understanding of data modeling, ETL processes, and data lakes.
• Experience with scripting languages such as Python or Scala is a plus.

Technologies & Tools

The Databricks Platform/DevOps Engineer will extensively work with technologies such as Apache Spark, Databricks workspace, AWS/Azure/Google Cloud services, Jenkins, Docker, and Kubernetes. The role also requires proficiency in scripting and programming languages like Python or Scala, along with relevant tools for data orchestration and monitoring.

Ideal Candidates

The ideal candidate for the Databricks Platform/DevOps Engineer position is someone who demonstrates a strong analytical mindset, with an aptitude for problem-solving and attention to detail. Candidates should possess excellent collaboration skills and the ability to communicate technical concepts to non-technical stakeholders effectively. A proactive approach to continuous learning and adaptation to evolving technological landscapes is essential, in addition to a commitment to delivering high-quality results in a fast-paced environment.