Data Platform Engineer with Databricks and Unity Catalog

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by GeorgiaTEK Systems Inc., so some details may not match this job. The employer's own page has the real description.

Position Overview

GeorgiaTEK Systems Inc. is seeking a highly skilled Data Platform Engineer with expertise in Databricks and Unity Catalog. This role involves designing, implementing, and maintaining scalable and robust data platforms to support data processing and analytics initiatives. The successful candidate will work collaboratively with data scientists, analysts, and other engineering teams to optimize data workflows, enhance data accessibility, and ensure data governance across the organization.

Job Responsibilities

• Collaborate with data teams to understand data requirements and translate them into technical specifications.
• Design, build, and optimize data pipelines using Databricks to ensure efficient data processing and storage.
• Implement and manage Unity Catalog for data governance and metadata management across various datasets.
• Monitor and troubleshoot data platforms to resolve performance issues and ensure high availability.
• Develop documentation and best practices for data platform architecture and data workflows.
• Conduct regular audits and maintain compliance with data security and governance policies.
• Stay updated with the latest developments in data engineering tools and technologies and recommend improvements.

Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, or a related field.
• Minimum of 3-5 years of experience in data engineering or a similar role, with a focus on Databricks and Unity Catalog.
• Proficiency in writing and optimizing SQL queries and understanding of data warehousing concepts.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Knowledge of data modeling, ETL processes, and data processing frameworks.
• Strong problem-solving capabilities and attention to detail.
• Ability to work collaboratively in a fast-paced, team-oriented environment.

Technologies & Tools

The Data Platform Engineer will primarily utilize Databricks for data processing and analytics, alongside Unity Catalog for data governance and metadata management. Familiarity with cloud services such as AWS or Azure will be essential, along with tools such as Apache Spark, SQL, and various ETL tools for effective data management. Additionally, knowledge of data visualization tools is advantageous.

Ideal Candidates

The ideal candidates for this position will possess a strong analytical mindset coupled with a creative approach to problem-solving. They should be proactive in identifying opportunities for data improvements and exhibit excellent communication skills. A commitment to learning and adaptability in the face of new technologies and methodologies will also mark a strong candidate, as will an eagerness to collaborate with cross-functional teams to achieve common goals.