Remote: Data Platform Engineer with Azure Databricks and Unity Catalog.

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Visionary Innovative Technology Solutions LLC, so some details may not match this job. The employer's own page has the real description.

Job Overview

Visionary Innovative Technology Solutions LLC is seeking a skilled Remote Data Platform Engineer specializing in Azure Databricks and Unity Catalog. In this role, you will be responsible for designing, implementing, and maintaining data solutions that enhance our data capabilities and analytics potential. You will collaborate with cross-functional teams to ensure data integrity and availability while fostering an environment of innovation through the latest technologies and best practices in data engineering.

Job Responsibilities

• Design, develop, and implement scalable data pipelines in Azure Databricks.
• Manage and optimize the Azure Unity Catalog for effective data governance and management.
• Collaborate with data scientists and analysts to support the delivery of data-driven insights.
• Ensure high performance of data systems by conducting regular testing and debugging.
• Implement data security measures to protect sensitive information in accordance with compliance regulations.
• Document engineering processes and report working progress to stakeholders.
• Stay updated with the latest industry trends and emerging technologies to improve existing systems.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• At least 3 years of experience working with Azure Databricks or similar data engineering platforms.
• Strong knowledge of Azure Unity Catalog and its features for data asset management.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with data modeling, ETL processes, and data warehousing principles.
• Familiarity with cloud architecture and best practices in data security and governance.
• Excellent problem-solving skills with the ability to work independently and in a team environment.

Technologies & Tools

This position utilizes a variety of technologies and tools including Azure Data Lake, Azure Databricks, Unity Catalog, SQL, Apache Spark, and CI/CD pipelines for efficient deployment and continuous integration of data solutions. Familiarity with tools such as Power BI for data visualization and Jupyter Notebooks for data analysis is also advantageous.

Ideal Candidates

The ideal candidates for this position will possess a strong analytical mindset and be driven by a passion for data. They should demonstrate excellent communication skills to convey technical information effectively to non-technical stakeholders. Additionally, a collaborative spirit and a commitment to continuous improvement will make them a valuable addition to our innovative team. Candidates who thrive in a fast-paced environment and are proactive in seeking solutions will excel in this role.