Databricks Engineer - Remote / Telecommute

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Dice, so some details may not match this job. The employer's own page has the real description.

Overview

The Databricks Engineer will be an integral part of our data engineering team, responsible for designing, building, and maintaining data pipelines and infrastructure solutions that leverage Databricks for big data processing and analytics. The successful candidate will support various data-intensive initiatives across the organization, collaborating closely with data scientists, analysts, and business stakeholders to ensure the integrity and availability of data. This role is 100% remote, offering flexibility and the opportunity to contribute to cutting-edge data solutions in a dynamic work environment.

Job Responsibilities

• Design and develop scalable data pipelines using Databricks and Apache Spark technologies.
• Implement ETL processes to ensure data is transformed and loaded efficiently.
• Collaborate with cross-functional teams to define data requirements and architecture.
• Optimize existing data processing workflows to enhance performance and reliability.
• Monitor and troubleshoot data pipeline issues to ensure high availability.
• Document architecture and data workflows to facilitate knowledge sharing and maintenance.
• Stay updated with industry best practices and trends related to big data and Databricks technologies.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of experience in data engineering or a similar role.
• Proficiency in Databricks, Apache Spark, and related big data technologies.
• Strong programming skills in languages like Python, Scala, or Java.
• Experience with SQL and data modeling practices.
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Knowledge of data warehousing concepts and techniques.

Technologies & Tools

The Databricks Engineer will utilize a variety of technologies and tools to effectively perform their role. Key technologies include Databricks for big data analytics, Apache Spark for large-scale data processing, and cloud services such as AWS, Azure, or Google Cloud for hosting data solutions. Additional tools may involve version control systems like Git for collaborative coding, and collaboration tools such as Slack or Microsoft Teams for team communication.

Ideal Candidates

The ideal Databricks Engineer candidate possesses strong analytical thinking and problem-solving skills, demonstrating the ability to manage complex data processes effectively. A proactive attitude towards learning new technologies and continuous improvement is essential. Excellent communication skills are crucial for collaborating with various teams and translating technical concepts into business strategies. Furthermore, candidates should exhibit attention to detail and a commitment to maintaining high standards of data quality and integrity.