100% Remote // Azure Data Brick engineer (PySpark, Python)

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Talent Groups, so some details may not match this job. The employer's own page has the real description.

Job Overview

Talent Groups is seeking a skilled Azure Data Brick Engineer with expertise in PySpark and Python for a 100% remote position. The selected candidate will be responsible for designing, developing, and maintaining data processing solutions that leverage Azure Data Bricks within the Azure cloud environment. This role requires a strong understanding of big data concepts, along with the ability to collaborate effectively with cross-functional teams to deliver high-quality data solutions that meet business requirements.

Job Responsibilities

• Design and implement scalable data processing solutions using Azure Data Bricks.
• Develop data pipelines that integrate with various Azure services and external data sources.
• Utilize PySpark and Python to transform and analyze large data sets.
• Collaborate with data engineers, data analysts, and stakeholders to document and refine data requirements.
• Optimize performance of existing data solutions and troubleshoot data processing issues.
• Ensure data integrity and accuracy throughout the data lifecycle.
• Participate in code reviews and contribute to best practices in software development.
• Stay current with industry trends and advancements in data processing technologies.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of professional experience in data engineering or analysis.
• Strong proficiency in PySpark and Python programming languages.
• Experience with Azure Data Bricks and Azure cloud services.
• Familiarity with big data concepts and technologies (Hadoop, Spark).
• Proficient in SQL and data modeling best practices.
• Excellent problem-solving abilities and attention to detail.
• Effective communication and collaboration skills.

Technologies & Tools

The Azure Data Brick Engineer will work extensively with the Microsoft Azure platform, employing tools such as Azure Data Bricks, Azure Data Lake, and Azure Blob Storage. Additionally, proficiency in open-source frameworks, including Apache Spark and Python libraries, will be crucial for transforming and managing large data sets efficiently. Experience in utilizing SQL for querying and manipulating data will also be integral to this position.

Ideal Candidates

Ideal candidates will possess a blend of technical proficiency and interpersonal skills, enabling them to thrive in a collaborative remote environment. They are expected to have a strong analytical mindset that drives them to seek innovative solutions to complex problems. A proactive approach to learning and adapting to new technologies, along with the ability to communicate effectively with diverse stakeholders, will set them apart. Additionally, a passion for data-driven decision making and attention to detail will contribute to their success in this role.