Job Description
Job Description
The client is strategically transitioning the Data Warehouse engineering operations onshore and is looking for a driven Data Warehouse Engineer to help lead this initiative.
Your primary focus will be migrating, building, and optimizing ELT pipelines that extract data from third-party APIs and load it into our data lake utilizing managed Delta tables. You will initially work closely with our existing offshore team to complete a comprehensive Knowledge Transfer (KT).
Key Responsibilities
ELT Pipeline Development: Design, build, and maintain robust ELT (Extract, Load, Transform) pipelines to migrate data from various third-party API sources into our enterprise data lake.
Databricks Ecosystem: Create, manage, and optimize managed Delta tables within a Databricks environment to ensure high data reliability and query performance.
API Integration: Securely connect to, authenticate with, and extract data from a wide variety of third-party RESTful APIs.
Data Modeling & Querying: Write clean, scalable, and efficient Python and SQL code to transform raw data into analytics-ready datasets.
Knowledge Transfer (KT): Actively participate in onboarding and KT sessions with the current offshore team to seamlessly transition ownership of existing pipelines and architecture.
Documentation: Maintain clear technical documentation for data pipelines, API integrations, and Delta table schemas to support future onshore hires.
Required Skills & Qualifications
Experience: 6+ years of experience in data engineering, data warehousing.
Core Languages: Strong, hands-on proficiency in Python and SQL.
Databricks & Data Lakes: Practical experience working with Databricks, specifically building and maintaining Delta tables and working within a modern data lake architecture.
API Expertise: Proven experience extracting and handling data from third-party APIs (handling pagination, rate limiting, JSON/XML parsing, etc.).
Methodology: Solid understanding of ELT methodologies and cloud-based data architecture.
Communication: Excellent verbal and written communication skills, with the ability to effectively absorb complex technical information during cross-border knowledge transfers.
The client is strategically transitioning the Data Warehouse engineering operations onshore and is looking for a driven Data Warehouse Engineer to help lead this initiative.
Your primary focus will be migrating, building, and optimizing ELT pipelines that extract data from third-party APIs and load it into our data lake utilizing managed Delta tables. You will initially work closely with our existing offshore team to complete a comprehensive Knowledge Transfer (KT).
Key Responsibilities
ELT Pipeline Development: Design, build, and maintain robust ELT (Extract, Load, Transform) pipelines to migrate data from various third-party API sources into our enterprise data lake.
Databricks Ecosystem: Create, manage, and optimize managed Delta tables within a Databricks environment to ensure high data reliability and query performance.
API Integration: Securely connect to, authenticate with, and extract data from a wide variety of third-party RESTful APIs.
Data Modeling & Querying: Write clean, scalable, and efficient Python and SQL code to transform raw data into analytics-ready datasets.
Knowledge Transfer (KT): Actively participate in onboarding and KT sessions with the current offshore team to seamlessly transition ownership of existing pipelines and architecture.
Documentation: Maintain clear technical documentation for data pipelines, API integrations, and Delta table schemas to support future onshore hires.
Required Skills & Qualifications
Experience: 6+ years of experience in data engineering, data warehousing.
Core Languages: Strong, hands-on proficiency in Python and SQL.
Databricks & Data Lakes: Practical experience working with Databricks, specifically building and maintaining Delta tables and working within a modern data lake architecture.
API Expertise: Proven experience extracting and handling data from third-party APIs (handling pagination, rate limiting, JSON/XML parsing, etc.).
Methodology: Solid understanding of ELT methodologies and cloud-based data architecture.
Communication: Excellent verbal and written communication skills, with the ability to effectively absorb complex technical information during cross-border knowledge transfers.
Most job openings for the size of the industry in August: professional and business services; private education and health care, 5 for every 100 jobs. Fewest: construction, 2.9. Compare industries →
View Similar Jobs
Matches Jobs
Similar jobs which you may be interested in. Typically using your existing skillset.
Senior Lead Threat Analyst (Black Lotus Labs)
Lumen Technologies.San Diego, USA
Full Time
Entry Level
3yr. Exp.
Enterprise Implementation Specialist, Americas
Ahrefs.San Diego, USA
Full Time
Entry Level
3yr. Exp.
Enterprise Implementation Specialist, Americas
Ahrefs.San Diego, USA
Full Time
Entry Level
3yr. Exp.
