Senior Data Engineer

In-Person Interview Mandatory

Job Summary:

We are seeking an experienced and results-driven Senior Data Engineer to lead the design, development, and optimization of our enterprise data architecture. In this role, you will build scalable batch and real-time data pipelines, design modern data warehouse/data lake solutions, and ensure high data quality and performance across our platform. You will collaborate closely with Data Scientists, Analytics Engineers, Software Developers, and Business Leadership to transform raw data into actionable insights.

Responsibilities:

• Design, construct, test, and maintain robust, scalable batch and streaming data pipelines (ETL/ELT) using technologies such as PySpark, Python, SQL, and Kafka.
• Build and optimize dimensional models, star/snowflake schemas, and data vault architectures in modern cloud platforms (e.g., Snowflake, Databricks, BigQuery, or Redshift).
• Optimize complex SQL queries, database indexing, partitioned data storage (Parquet, Delta Lake, Iceberg), and distributed compute jobs for speed and cost efficiency.
• Implement automated testing, monitoring, and alerting frameworks (e.g., Great Expectations, dbt tests) to ensure data integrity, schema validation, and SLA adherence.
• Automate pipeline deployment, infrastructure provision (IaC with Terraform/CloudFormation), and orchestration workflows using tools like Apache Airflow, Prefect, or Dagster.
• Mentor junior and mid-level data engineers, participate in code reviews, drive architectural standards, and partner with cross-functional product and analytics teams.

Required Skills:

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related technical field (or equivalent practical experience).
• 9 to 12 years of hands-on experience in data engineering, data warehousing, and software engineering.
• Advanced proficiency in Python and SQL (Scala or Java is a plus).
• Hands-on experience with at least one major cloud provider (AWS, GCP, or Azure).
• Proven experience with distributed processing engines (Apache Spark / PySpark).
• Hands-on experience with workflow orchestration platforms (Apache Airflow, Prefect, or MWAA).
• Deep expertise in Snowflake, Databricks, BigQuery, or AWS Redshift.
• Experience with dbt (data build tool) for SQL-based transformations and testing.