Full Time

Senior AWS Data Engineer – PySpark, Glue & Redshift

Delta Computer Consulting
Full-time
Mid-Senior level
IT Services and IT Consulting
Los Angeles Metropolitan Area
$145,000 - $210,000* / year (about $70–$101 an hour)

Job Description


Compensation & Location

Location: Torrance, CA

Schedule: Full-time contract

Work Type: Onsite/Hybrid schedule not specified

Pay Range: $81.00 – $88.00 per hour (based on experience & internal equity)

Benefits: Health, Dental, Vision, Life Insurance, 401k, Continuing & On-Demand Education

📍📍 Must be willing to work in Torrance, CA

🌐🌐 Sponsorship is available for this role


🚫🚫 W2 ONLY — No C2C, No 3rd Parties


Position Summary

We are seeking an experienced Senior AWS Data Engineer to design, develop, and optimize scalable cloud-based data pipelines supporting enterprise data integration, warehousing, and analytics initiatives.

This role requires strong hands-on experience with AWS Glue, EMR, S3, Redshift, Athena, MWAA (Airflow), Python, PySpark, and Apache Spark, along with expertise in ETL/ELT development, distributed data processing, and cloud data architecture.

The ideal candidate has 7+ years of data engineering experience, including extensive AWS data engineering expertise, strong programming skills, and experience implementing secure, reliable, high-performance data pipelines.

Candidates should also have experience with Informatica IDMC/Informatica Cloud, AWS IAM, Lake Formation, CI/CD, and Generative AI or Agentic AI use cases in data engineering.

Key Responsibilities

  • Design, develop, and maintain scalable ETL/ELT pipelines using AWS Glue, EMR, Lambda, Redshift, and MWAA (Apache Airflow).
  • Build and optimize large-scale data processing solutions using Python, PySpark, Apache Spark, and distributed computing frameworks.
  • Develop data integration workflows that efficiently extract, transform, and load data into enterprise data warehouses and data lakes.
  • Work with Amazon S3, Redshift, Athena, RDS, Redshift Spectrum, and other AWS data services.
  • Implement data pipeline orchestration, scheduling, monitoring, and automated error handling.
  • Optimize Redshift queries, database performance, data transformations, and cloud resource utilization.
  • Improve pipeline performance, scalability, reliability, and cost efficiency while meeting established SLAs.
  • Implement data quality checks, validation rules, cleansing processes, and monitoring mechanisms.
  • Develop secure data access solutions using AWS IAM, Lake Formation, IAM roles, policies, and access controls.
  • Support enterprise data lake security, governance, and regulatory compliance requirements.
  • Implement CI/CD pipelines and DevOps practices for automated deployment and release of data engineering solutions.
  • Develop and maintain integrations using Informatica IDMC/Informatica Cloud.
  • Support Generative AI and Agentic AI use cases involving data engineering workflows and data platforms.
  • Collaborate with data analysts, business stakeholders, and technical teams to translate business requirements into production-ready pipelines.
  • Ensure timely, accurate, and reliable data availability for business intelligence and analytics.
  • Maintain technical documentation, troubleshoot production issues, and continuously improve data integration processes.


Required Skills & Experience

  • 7+ years of experience in data engineering, database design, ETL/ELT development, and data warehousing.
  • 5+ years of hands-on AWS experience, including S3, EMR, Glue, Athena, Redshift, RDS, Redshift Spectrum, and Airflow.
  • 5+ years of programming experience using Python, PySpark, or Scala.
  • Strong experience with Apache Spark, PySpark, Hadoop, and distributed data processing.
  • Hands-on experience building and maintaining AWS Glue ETL jobs, EMR workloads, and Airflow/MWAA orchestration pipelines.
  • Strong experience with Amazon Redshift, including SQL optimization, performance tuning, and large-scale data warehousing.
  • Experience designing and maintaining scalable data lakes and cloud-based data integration solutions.
  • 2+ years of CI/CD and DevOps experience supporting automated builds, deployments, and releases of data pipelines.
  • 2+ years of hands-on AWS IAM and AWS Lake Formation experience, including roles, policies, access controls, and data lake governance.
  • Experience with Informatica IDMC or Informatica Cloud.
  • Experience with Generative AI or Agentic AI use cases in data engineering.
  • Strong knowledge of data modeling, database design, SQL, and ETL/ELT processes.
  • Experience implementing data validation, data cleansing, data quality monitoring, and error handling.
  • Ability to troubleshoot pipeline failures, optimize data workflows, and improve performance and cost efficiency.
  • Experience translating business requirements into technical specifications and production-ready data solutions.
  • Strong understanding of cloud security, data governance, and regulatory compliance.
  • Excellent analytical, troubleshooting, communication, and cross-functional collaboration skills.


Desired Skills

  • Experience supporting large-scale enterprise AWS data platforms and cloud data modernization initiatives.
  • Experience with AWS Lambda, AWS MWAA, Redshift Spectrum, and advanced Redshift optimization.
  • Experience with infrastructure automation and CI/CD tools such as Terraform, Jenkins, GitHub Actions, or AWS CodePipeline.
  • Knowledge of data lake architecture, lakehouse concepts, and modern cloud data platforms.
  • Experience optimizing distributed Spark workloads and managing large datasets.
  • Experience with Informatica Intelligent Data Management Cloud (IDMC) integrations and cloud data pipelines.
  • Experience applying Generative AI, LLMs, or Agentic AI to data engineering workflows, automation, or data quality processes.
  • Familiarity with data cataloging, metadata management, data lineage, and enterprise data governance.
  • Experience supporting business intelligence, reporting, and advanced analytics platforms.
  • Experience working in large enterprise, automotive, or manufacturing environments.


Your LinkedIn About section is where you say what you do and what you are looking for. Type your job and a few facts, and pick from five drafts. Write an About section →

View Similar Jobs

Matches Jobs

Similar jobs which you may be interested in. Typically using your existing skillset.