Remote: Data Platform Engineer with Azure Databricks and Unity Catalog.

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Visionary Innovative Technology Solutions LLC, so some details may not match this job. The employer's own page has the real description.

Job Overview

Visionary Innovative Technology Solutions LLC is seeking a highly skilled and motivated Data Platform Engineer to join our remote team. The ideal candidate will leverage Azure Databricks and Unity Catalog to design and implement robust data engineering solutions, ensuring high-performance data processing and seamless integration across our data platforms. This role demands a proactive approach to shaping data architecture while collaborating with cross-functional teams to facilitate data-driven decision-making within the organization.

Job Responsibilities

• Design, develop, and maintain data pipelines utilizing Azure Databricks for efficient processing and analysis of large datasets.
• Implement and manage Unity Catalog for data governance, security, and organization of metadata across the data ecosystem.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and provide tailored solutions.
• Optimize data processing performance and scalability by applying best practices in data engineering methodologies.
• Monitor and troubleshoot data processing workflows to ensure reliability and data accuracy.
• Stay current with technology trends in cloud computing and data engineering to drive innovation within data practices.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 3+ years of experience working with Azure Databricks and Unity Catalog.
• Proficient in SQL and programming languages such as Python and Scala.
• Strong understanding of data modeling, ETL processes, and data warehousing concepts.
• Experience with Azure cloud services and familiarity with data governance principles.
• Ability to work collaboratively in a remote team environment, demonstrating strong communication and problem-solving skills.

Technologies & Tools

The Data Platform Engineer will work primarily with Azure Databricks and Unity Catalog, alongside a suite of tools including Azure Data Factory, Power BI, and Snowflake for data integration and analytics. Familiarity with Git for version control and CI/CD pipelines will also be beneficial in executing project deliverables efficiently.

Ideal Candidates

Ideal candidates will possess a strong analytical mindset coupled with a passion for leveraging data to drive business improvements. They are expected to be detail-oriented, adaptable, and proactive in their approach to problem-solving. The successful candidate should also exhibit excellent interpersonal skills, fostering collaboration within a diverse team and contributing to a positive remote work culture.