Sr. Data Platform Engineer (REMOTE) - LLM, AZURE Infrastructure, Databricks, ADLS, PostgresDB, Event Hubs, AKS, Databricks Admin, Unity Catalog, Python. Python + SQL, MLOps, CI/CD, GitHub

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by LingaTech, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Sr. Data Platform Engineer at LingaTech will play a critical role in designing, building, and optimizing our data infrastructure while leveraging cutting-edge technologies. This remote position is responsible for managing data workflows across LLM, Azure infrastructure, Databricks, and other key data technologies. The ideal candidate will collaborate closely with data scientists and engineers to create scalable solutions that enhance our data ecosystem, facilitating robust analytics, machine learning applications, and operational efficiency.

Job Responsibilities

• Design and implement data workflows utilizing Azure infrastructure, Databricks, and ADLS.
• Manage and optimize PostgreSQL databases and ensure data integrity and security.
• Develop and maintain data pipelines using Python and SQL in alignment with MLOps best practices.
• Administer Databricks workspaces and manage resources within the Unity Catalog.
• Monitor and troubleshoot data processing issues and recommend enhancements for performance improvements.
• Implement CI/CD processes to automate deployment workflows in GitHub.
• Collaborate with cross-functional teams to support evolving data needs and ensure alignment with business objectives.
• Leverage Event Hubs and AKS for real-time data processing and orchestration of data workflows.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Minimum of 5 years of experience in data engineering or data platform engineering roles.
• Strong proficiency in Python and SQL for data manipulation and analysis.
• Experience with Azure Cloud services, particularly with Databricks and ADLS.
• Familiarity with PostgreSQL or similar relational databases.
• Hands-on experience with CI/CD tools and methodologies, particularly in a GitHub environment.
• Knowledge of MLOps principles and implementation strategies.
• Strong problem-solving skills and the ability to work collaboratively across teams.

Technologies & Tools

The Sr. Data Platform Engineer will work extensively with technologies including Azure cloud services, Databricks, Azure Data Lake Storage (ADLS), PostgreSQL, Event Hubs, Azure Kubernetes Service (AKS), and the Unity Catalog. The role also involves using programming languages such as Python and SQL, along with tools for continuous integration and deployment (CI/CD), specifically within the GitHub ecosystem.

Ideal Candidates

Ideal candidates for this position will possess a blend of technical expertise and strong interpersonal skills. They will be proactive problem-solvers who thrive in a fast-paced environment and are committed to continuous improvement and innovation. Strong analytical and critical thinking capabilities, along with a collaborative mindset, are essential for working effectively with cross-functional teams and meeting organizational goals.