Databricks Data Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Smart IT Frame LLC, so some details may not match this job. The employer's own page has the real description.

Job Overview

Smart IT Frame LLC is seeking a skilled Databricks Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing, building, and maintaining data solutions that leverage the Databricks platform. This role involves collaborating with data scientists and analysts to develop scalable data pipelines and ensure data is accessible and actionable. A focus on optimizing performance, ensuring data quality, and integrating diverse data sources into a unified data architecture is essential. This is an exciting opportunity to contribute to impactful data-driven decisions within a rapidly evolving environment.

Job Responsibilities

• Design and implement robust data pipelines using Databricks and Apache Spark.
• Collaborate with cross-functional teams to determine data requirements and develop optimal data strategies.
• Monitor and troubleshoot performance issues within data solutions to ensure high availability and reliability.
• Develop and maintain ETL processes to facilitate the integration of various data sources into the data ecosystem.
• Ensure data quality and integrity through comprehensive testing and validation methodologies.
• Implement data governance policies and best practices for data lifecycle management.
• Document processes and architecture to facilitate knowledge sharing and compliance with organizational standards.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of experience in data engineering or related roles, with a strong focus on data processing technologies.
• Proficient in Apache Spark and the Databricks platform.
• Strong programming skills in languages such as Python, Scala, or Java.
• Familiarity with data warehousing solutions and cloud technologies (e.g., AWS, Azure, Google Cloud).
• Experience with SQL and NoSQL databases, data modeling, and query optimization.
• Excellent problem-solving skills and attention to detail, with the ability to manage multiple tasks effectively.

Technologies & Tools

The Databricks Data Engineer position at Smart IT Frame LLC will utilize a range of cutting-edge technologies and tools including Apache Spark, Databricks, SQL databases, NoSQL databases, and cloud platforms such as AWS and Azure. Familiarity with tools for data integration, ETL processes, and data visualization such as Tableau or Power BI may also be beneficial. The focus will be on building efficient data architectures that support advanced analytics and machine learning initiatives.

Ideal Candidates

The ideal candidate for the Databricks Data Engineer position possesses a strong analytical mindset and a genuine passion for data-driven solutions. They should have excellent communication skills to effectively collaborate with technical and non-technical stakeholders. Demonstrated adaptability and a proactive approach to learning new technologies in a fast-paced environment will be essential. A strong commitment to data quality and best practices in data engineering will also distinguish top candidates.