Databricks/DevOps Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Volto Consulting, so some details may not match this job. The employer's own page has the real description.

Job Overview

Volto Consulting is seeking a highly skilled Databricks/DevOps Engineer to join our innovative team. The ideal candidate will play a pivotal role in enhancing our data processing and deployment environments using Databricks Unified Analytics Platform. Primary responsibilities include optimizing the integration of various data sources, managing cloud-based infrastructures, automating deployment processes, and collaborating with data scientists and analysts to ensure seamless data flow. This position demands a strong understanding of both Databricks and DevOps methodologies to facilitate scalable data solutions and support continuous integration and delivery practices.

Job Responsibilities

• Design, implement, and maintain Databricks-based data pipelines and workflows.
• Automate deployment processes for infrastructure using tools such as Azure DevOps or Jenkins.
• Monitor and optimize data processing jobs to ensure efficiency and reliability.
• Collaborate with data scientists and analytics teams to support data ingestion and transformation processes.
• Implement best practices for data governance, security, and compliance while working with cloud technologies.
• Provide troubleshooting and support for production-level services and data environments.
• Develop and maintain documentation for systems, processes, and technical specifications.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in a DevOps or Data Engineering role, with hands-on experience in Databricks.
• Proficiency in Python, SQL, and Spark with extensive experience in data processing technologies.
• Experience with cloud providers such as Azure, AWS, or Google Cloud Platform.
• Understanding of CI/CD methodologies and tools (e.g., Jenkins, GitHub Actions).
• Knowledge of containerization tools such as Docker and orchestration frameworks like Kubernetes.
• Strong analytical, problem-solving, and troubleshooting skills.

Technologies & Tools

The Databricks/DevOps Engineer will primarily work with Databricks Unified Analytics Platform for data engineering tasks. Additionally, proficiency in cloud platforms like Microsoft Azure, Amazon Web Services (AWS), or Google Cloud Platform is expected. Familiarity with automation and CI/CD tools such as Jenkins, Git, and containerization technologies like Docker, as well as orchestration platforms like Kubernetes, is also crucial for this role.

Ideal Candidates

The ideal candidate for this position is a detail-oriented and proactive professional who thrives in a dynamic, collaborative environment. They should possess excellent communication skills, both verbal and written, to effectively interact with diverse teams. This person should have a passion for data engineering and DevOps practices, with a strong curiosity to learn new technologies and improve existing processes. Additionally, a commitment to quality and continuous improvement, along with a problem-solving mindset, will significantly contribute to their success at Volto Consulting.