Databricks Data Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by ALOIS Solutions, so some details may not match this job. The employer's own page has the real description.

Job Overview

At ALOIS Solutions, we are seeking a skilled Databricks Data Engineer who will be instrumental in executing data engineering tasks that enhance the performance and reliability of our analytics solutions. The selected candidate will be responsible for designing, building, and transforming data pipelines, ensuring data accuracy, and optimizing processes for effective data consumption. The role demands strong collaboration with cross-functional teams to align data solutions with business requirements and leverage advanced analytical techniques to drive data-based decision-making.

Job Responsibilities

• Develop and manage robust data pipelines using Databricks and Apache Spark to streamline data processing workflows.
• Ensure the quality and integrity of data by performing thorough data validation and cleansing processes.
• Collaborate with data scientists and analysts to define data requirements and support analytical initiatives.
• Optimize existing ETL processes for improved efficiency and scalability.
• Implement best practices in data storage, data management, and engineering workflows.
• Monitor and troubleshoot data systems to ensure smooth operation and timely data delivery.
• Document workflows, processes, and data architecture to support team knowledge sharing.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years' experience in data engineering or related roles.
• Proficiency in Apache Spark, PySpark, and Databricks.
• Strong SQL skills with advanced knowledge of database systems (e.g., MySQL, PostgreSQL, or equivalent).
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Familiarity with data modeling, data warehousing concepts, and data governance principles.
• Ability to work effectively in a collaborative team environment.

Technologies & Tools

The Databricks Data Engineer role will utilize a range of technologies and tools, including but not limited to Databricks, Apache Spark, SQL databases, cloud platforms (such as AWS, Azure, or GCP), and data integration tools. Proficiency in programming languages such as Python and Scala will also be essential to ensure efficient data processing and transformation.

Ideal Candidates

The ideal candidate for the Databricks Data Engineer position will demonstrate a strong analytical mindset, attention to detail, and a passion for data-driven decision-making. They should possess excellent problem-solving skills and the ability to work under tight deadlines and changing priorities. Effective communication and interpersonal skills are crucial for collaborating with IT and business teams, while a proactive approach to self-learning and adopting new technologies will drive innovation within our organization.