Cloud DataBricks Engineer-Azure

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by BigRio, so some details may not match this job. The employer's own page has the real description.

Job Overview

BigRio is seeking a skilled Cloud DataBricks Engineer-Azure to join our dedicated team. The ideal candidate will be responsible for designing, implementing, and optimizing data processing solutions using Cloud DataBricks and Azure services. This role entails collaborating with cross-functional teams to enhance the data analytics pipeline while ensuring high availability, performance, and adherence to data governance. The successful candidate will leverage their technical expertise to contribute to the organization’s cloud adoption strategy and drive business insights through efficient data management practices.

Job Responsibilities

• Design and implement scalable data processing frameworks using Azure DataBricks.
• Collaborate with data scientists and analysts to create robust data pipelines that meet analytical requirements.
• Optimize existing data processes for performance and cost efficiency in Azure cloud environments.
• Monitor and troubleshoot data workflows to ensure seamless data integration and availability.
• Develop and maintain documentation for data architecture, pipelines, and best practices.
• Stay up to date with the latest Azure and DataBricks features and functionalities to enhance the data engineering process.
• Implement security measures to maintain data integrity and compliance with industry standards.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or related field.
• 4+ years of experience in data engineering or related roles.
• Strong expertise in Azure DataBricks, Azure Data Lake, and Azure Synapse Analytics.
• Proficient in programming languages such as Python, Scala, or Java.
• Experience with data modeling, ETL processes, and database management systems.
• Knowledge of data visualization tools such as Power BI or Tableau is a plus.
• Excellent problem-solving skills and ability to work in a fast-paced environment.
• Strong communication skills for effective collaboration with technical and non-technical teams.

Technologies & Tools

The Cloud DataBricks Engineer will primarily utilize technologies and tools such as Azure DataBricks, Azure Data Lake Storage, Azure Synapse, Apache Spark, and SQL databases. Familiarity with DevOps tools for CI/CD processes, as well as data governance and security frameworks, will also be essential to performing the responsibilities of this position efficiently.

Ideal Candidates

The ideal candidate for the Cloud DataBricks Engineer-Azure role at BigRio is someone who demonstrates a strong commitment to continuous learning and professional growth. They should possess analytical thinking, creative problem-solving abilities, and a collaborative spirit to efficiently work in team settings. A proactive approach towards enhancing operational efficiency and the ability to adapt to changing technologies and business needs are critical success factors for this role. Furthermore, the candidate should exhibit excellent interpersonal skills, allowing for effective communication and relationship-building across various departments.