Data Engineer (Databricks)

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Recruiters Workforce, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Data Engineer at Recruiters Workforce will play a critical role in designing, building, and maintaining scalable data processing systems and architectures. This position requires expertise in Databricks as the primary data engineering platform to facilitate efficient data management and processing. The selected candidate will collaborate with data scientists, analysts, and other stakeholders to ensure data integrity and availability, enabling data-driven decision-making across the organization.

Job Responsibilities

• Design and implement data pipelines using Databricks to aggregate, transform, and load data from various sources.
• Maintain and optimize ETL processes for data ingestion, cleansing, and transformation.
• Collaborate with cross-functional teams to identify data requirements and deliver actionable insights.
• Develop and maintain data models, ensuring data quality and consistency across platforms.
• Implement data governance policies and practices to ensure compliance and security.
• Monitor and troubleshoot data processing workflows for performance issues and data discrepancies.
• Document data architecture, processes, and procedures to facilitate knowledge sharing and onboarding.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or related field.
• Proven experience as a Data Engineer or similar role, with a focus on data processing and ETL.
• Proficiency in Databricks, Apache Spark, SQL, and Python.
• Experience with data warehousing concepts, architectures, and solutions.
• Strong understanding of data modeling techniques and best practices.
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

The Data Engineer will work extensively with Databricks for data processing and analysis, leveraging its integrated workspace to collaborate with other team members. Additional technologies and tools include Apache Spark, SQL databases, cloud platforms (AWS/Azure/Google Cloud), data governance tools, and data visualization software. Familiarity with these technologies is essential for the successful execution of data engineering tasks and achieving business objectives.

Ideal Candidates

Ideal candidates for the Data Engineer position will exhibit strong analytical and technical skills alongside exceptional attention to detail. They should possess the ability to work in a fast-paced environment and effectively manage multiple projects while collaborating with diverse teams. A proactive approach to problem-solving and a passion for data-driven decision-making are essential characteristics. Furthermore, candidates who continuously seek opportunities for personal and professional growth will thrive in our dynamic environment at Recruiters Workforce.