Azure Databricks Engineer

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by GAC Solutions, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Azure Databricks Engineer at GAC Solutions plays a crucial role in leveraging Databricks to enhance productivity and streamline data workflows. This position is responsible for designing and implementing data solutions that utilize Azure Databricks, ensuring optimal performance, and collaborating with cross-functional teams to support data-driven decision-making. The incumbent will be tasked with managing ETL processes, optimizing data processing pipelines, and contributing to the overall strategy for data integration and analytics within the organization.

Job Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Databricks.
• Collaborate with data scientists and analysts to understand their data needs and architect solutions that fulfill those requirements.
• Implement best practices for data ingestion, transformation, and storage.
• Monitor and troubleshoot data workflows to ensure seamless operation and minimal downtime.
• Work with Azure Data Services, including Azure Data Lake Storage, Azure SQL Database, and Azure Synapse Analytics.
• Optimize existing data processing pipelines for performance improvements.
• Document data workflows, architecture, and coding standards.
• Stay updated on industry trends and emerging technologies related to data engineering and analytics.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of experience in data engineering or related roles.
• Proficient in Azure Databricks with hands-on experience in Spark and data lake concepts.
• Strong programming skills in Python or Scala, with familiarity in SQL.
• Experience with ETL processes and tools.
• Knowledge of cloud architecture and services in the Azure ecosystem.
• Ability to work independently and collaboratively in a fast-paced environment.
• Excellent problem-solving skills and attention to detail.

Technologies & Tools

The Azure Databricks Engineer will utilize a variety of technologies and tools, including but not limited to Azure Databricks, Apache Spark, Azure Data Lake Storage, Azure SQL Database, Azure Synapse Analytics, and relevant ETL and data transformation tools. The role also requires a functioning knowledge of programming languages such as Python and Scala for data processing tasks.

Ideal Candidates

Ideal candidates for the Azure Databricks Engineer position are those who exhibit a strong analytical mindset coupled with technical expertise in data engineering. They should possess exceptional communication skills to effectively collaborate with various stakeholders and demonstrate a proactive approach to problem-solving. The ability to adapt to new challenges, an eagerness to learn and grow professionally, and a commitment to delivering high-quality results will also distinguish the right candidate for this role.