Job Description
We are seeking an experienced Senior Data Engineer / Databricks Specialist, Remote. Pl. go through the below description and let me know your interest. If you are interested, kindly share a copy of your resume to [email protected] along with the rate / salary and best time to reach you.
Key Roles And Responsibilities
In addition to the mandatory Databricks Data Engineer certification, candidates holding any of the following will be prioritized:
Key Roles And Responsibilities
- Develop and optimize scalable Spark applications in AWS Databricks using Python, PySpark and SQL to support data processing and analytics needs.
- Design, build and maintain robust ETL/ELT pipelines and implement Medallion Architecture (Bronze, Silver, Gold layers), ensuring data integrity, completeness and lineage.
- Administer Databricks workspaces, including cluster provisioning, job scheduling, access controls, Unity Catalog and notebook management.
- Manage AWS cloud infrastructure assets including S3, Lambda, IAM, Aurora and Glue, and provision environments using Terraform Infrastructure-as-Code (IaC).
- Perform query optimization, performance tuning, troubleshooting and debugging for complex Spark jobs and large-scale data solutions.
- Merge, consolidate and process structured, semi-structured and unstructured data sets from diverse API endpoints and databases.
- Embed security gates and compliance controls within DevSecOps CI/CD pipelines (Jenkins, GitLab CI) to support FISMA/FedRAMP requirements, ATO reassessments and POA&M resolution.
- Implement rigorous data quality controls, anomaly detection, validation rules and automated lineage tracking across all pipelines.
- Collaborate within cross-functional Agile/Scrum teams in multi-contractor federal environments, providing technical leadership and production on-call support.
- Citizenship: U.S. citizenship required.
- Mandatory certification: Active Databricks Certified Data Engineer Associate or Databricks Certified Data Engineer Professional.
- Experience: 6+ years of dedicated hands-on experience in Databricks development.
- Core big data stack: Production proficiency with PySpark, Spark SQL, Apache Spark and Delta Lake.
- Cloud ecosystem: Strong experience in AWS Cloud services including S3, IAM, Lambda, Glue and Aurora.
- Automation and DevOps: Production experience with Terraform (IaC) and CI/CD tools (Jenkins, GitLab CI or equivalent).
- Languages and frameworks: Deep proficiency in Python and SQL; working knowledge of REST APIs for ingestion.
- Clearance: Ability to obtain and maintain a Public Trust.
- Methodology and education: Hands-on experience in Agile/Scrum delivery environments. Bachelor's degree in CS, IT, Data Science or a related engineering discipline.
- Hands-on experience with Unity Catalog, Delta Live Tables (DLT) and AI/ML pipeline support (MLflow, Model Registry).
- Experience in Federal Government contracting and compliance frameworks (FISMA / FedRAMP environments).
- Active security clearance (Public Trust or higher).
In addition to the mandatory Databricks Data Engineer certification, candidates holding any of the following will be prioritized:
- Databricks Certified Machine Learning Associate
- AWS Certified Solutions Architect Associate or Professional
- AWS Certified Data Engineer Associate
Worldwide, unemployment was 4.9% in 2025, and 12.4% among 15 to 24-year-olds, by the International Labour Organization’s estimates. Every country, ranked →
View Similar Jobs
Matches Jobs
Similar jobs which you may be interested in. Typically using your existing skillset.