Databricks Architect

Required Skills & Experience

• Current, hands-on development experience is essential. This role requires the ability to read, write, debug, and lead code reviews of production PySpark and SQL code—not just architectural oversight. Candidates must demonstrate recent (within past 12 months) hands-on development work including debugging live code, explaining business logic and technical implementation, optimizing queries, and implementing data pipelines. Ability to comprehend and explain unfamiliar code samples is a core requirement.

• Advanced proficiency in SQL and Python/PySpark with demonstrated ability to write complex transformations, optimize query performance, explain code logic at both business and technical levels, and troubleshoot production issues. Must be comfortable with CTEs, window functions, table-valued functions, APPLY operators, advanced SQL operators, and PySpark DataFrame/RDD operations. Current, active coding skills required—not aspirational or theoretical knowledge.

• 7+ years of experience in data architecture, data engineering, or platform engineering roles, with at least 3+ years focused on Databricks platform architecture.

• Expert-level knowledge of Databricks platform components: Unity Catalog, Delta Lake, Delta Live Tables, Workflows, SQL Warehouses, MLflow, and Databricks SQL.

• Deep expertise in Unity Catalog governance, including metastore design, catalog/schema strategies, permission models, data lineage, and multi-workspace/multi-cloud patterns.

• Strong architectural background in cloud platforms (Azure, AWS, or GCP), including storage services, identity management (Azure AD, AWS IAM), networking, and security best practices.

• Proven experience designing enterprise-scale data architectures, including medallion/multi-hop architectures, data mesh patterns, domain-driven design, and data product frameworks.

• Hands-on experience with infrastructure-as-code (Terraform, ARM templates, CloudFormation) for platform configuration and governance automation.

• Strong understanding of DevOps practices, CI/CD pipelines, version control strategies, and automated testing for data platforms.

• Experience with performance tuning, cost optimization, and capacity planning for large-scale data platforms.