Databricks Platform Engineer

Databricks Platform Engineer

📍 Location: Columbus, Ohio, United States

🏢 Industry: Hospitals and Health Care

💼 Work Setting: Hybrid

Are you passionate about building and optimizing modern cloud data platforms that power enterprise analytics and data-driven decision-making? We are seeking a Databricks Platform Engineer to support the scalability, reliability, and security of a cloud-based data ecosystem. In this role, you will leverage your expertise in platform administration, cloud infrastructure, automation, and data engineering to enable high-quality data solutions while collaborating with cross-functional teams to enhance operational excellence.

Key Responsibilities

Data Platform Operations

• Administer, maintain, and optimize Databricks environments supporting analytics and data engineering workloads.
• Manage platform configurations, cluster performance, capacity planning, and operational stability.
• Monitor platform health and troubleshoot performance, reliability, and operational issues.
• Support the lifecycle management of platform resources across development, testing, and production environments.

Cloud Infrastructure & Automation

• Develop and maintain Infrastructure as Code (IaC) solutions using Terraform or similar tools.
• Support and optimize cloud-based data infrastructure within enterprise cloud environments.
• Implement automation solutions to improve platform reliability, consistency, and operational efficiency.
• Collaborate with infrastructure and security teams to align with organizational standards and best practices.

Data Engineering Enablement

• Build and support data ingestion frameworks and pipelines that facilitate reliable data movement across systems.
• Partner with Data Engineers to improve platform capabilities and optimize large-scale data processing workloads.
• Support structured and unstructured data workloads in cloud-native environments.
• Identify and resolve performance bottlenecks impacting data processing and accessibility.

Governance & Security

• Support enterprise data governance initiatives and platform governance capabilities.
• Implement and maintain data access controls, permissions, and security standards.
• Ensure compliance with data management, security, and governance requirements.
• Contribute to metadata management, data lineage, and data quality initiatives.

Continuous Improvement

• Evaluate emerging technologies and platform enhancements to drive innovation.
• Recommend and implement improvements that increase scalability, reliability, and developer productivity.
• Document platform standards, operational procedures, and support processes.
• Participate in incident response, root cause analysis, and operational reviews.

Required Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience.
• 4+ years of experience in Data Engineering, Data Platform Engineering, Cloud Engineering, or a related discipline.
• Hands-on experience administering Databricks environments.
• Strong experience with Terraform and Infrastructure as Code practices.
• Experience supporting cloud infrastructure and services.
• Experience developing and maintaining data ingestion and integration solutions.
• Strong SQL, troubleshooting, and problem-solving skills.
• Experience supporting production environments and operational workloads.
• Solid understanding of data modeling, data governance, and modern data architecture principles.

Preferred Qualifications

• Experience with data governance and catalog solutions within modern data platforms.
• Experience with cloud data technologies such as data lakes, data warehouses, or analytics platforms.
• Proficiency in Python and automation for data engineering workloads.
• Experience working in large-scale cloud analytics environments.
• Knowledge of CI/CD pipelines and deployment automation practices.
• Experience working within highly regulated industries.

Education

• Bachelor's degree in Computer Science, Information Technology, Engineering, Information Systems, or a related field.