Cloud Engineer

ob Description: Cloud Data Engineer – AWS / Databricks / Power BI

Location: remote

Experience: 8+ Years

Job Summary

We are looking for a hands-on cloud data engineer with strong experience in Databricks, AWS Cloud, Python, AWS Lambda, Amazon S3, and MongoDB. The ideal candidate will develop and maintain cloud-based data pipelines, integrate multiple data sources, and deliver business intelligence solutions using Power BI and/or Tableau.

The candidate should be familiar with GitHub, GitHub Actions CI/CD pipelines, and Agile development methodologies.

This is a hands-on development role, not an architect or management position. Candidates must have recent coding experience and the ability to build, test, troubleshoot, and maintain data engineering solutions.

Primary Technical Skills

• Databricks: Data processing, notebook development, and data pipeline implementation.
• AWS Cloud: Cloud-based data processing and application integration.
• Python / PySpark: Data transformation, ETL/ELT development, and automation.
• AWS Lambda: Serverless functions and event-driven processing.
• Amazon S3: Data storage, ingestion, and integration with data pipelines.
• MongoDB: Database queries, data extraction, and integration with data processing workflows.
• Power BI: Data modeling, DAX, Power Query, dashboard development, and reporting.
• GitHub & GitHub Actions: Version control, automated builds, testing, and CI/CD pipelines.
• Agile Development: Sprint planning, daily stand-ups, code reviews, and collaborative delivery.

BI requirement: Power BI is mandatory at minimum. Experience with both Power BI and Tableau is preferred.

Key Responsibilities

1. Cloud Data Engineering

• Develop, test, and maintain scalable data pipelines using Databricks, Python, and AWS services.
• Build data ingestion and transformation processes for structured and semi-structured data.
• Process datasets stored in Amazon S3 and integrate data from MongoDB.
• Develop AWS Lambda functions for data processing, automation, and event-driven workflows.
• Implement data validation, error handling, and monitoring.
• Troubleshoot data pipeline failures and optimize processing performance.

2. Databricks and Python Development

• Develop Databricks notebooks, jobs, and data transformation workflows.
• Use Python and PySpark to cleanse, transform, and process large datasets.
• Build reusable ETL/ELT components and implement data quality checks.
• Optimize Spark jobs and improve data processing efficiency.
• Maintain reliable, scalable, and production-ready data pipelines.

3. Business Intelligence and Reporting

• Develop and maintain Power BI dashboards, reports, and data models.
• Create DAX measures and use Power Query for data transformation.
• Connect BI tools to curated datasets and appropriate cloud data sources.
• Translate business requirements into meaningful reports and visualizations.
• Validate reporting accuracy and resolve data discrepancies.
• Support Tableau dashboard development where required.

4. CI/CD and Agile Development

• Use GitHub for source control, branching, pull requests, and code reviews.
• Develop and maintain automated CI/CD workflows using GitHub Actions.
• Support automated testing, deployment, and release management.
• Collaborate with data engineers, BI developers, QA engineers, and business stakeholders.
• Participate in Agile ceremonies, including sprint planning, daily stand-ups, and retrospectives.

Required Qualifications

• 8+ years of overall IT experience, preferably in cloud and data engineering.
• Hands-on experience with Databricks, AWS Cloud, Python, AWS Lambda, Amazon S3, and MongoDB.
• Strong experience building and maintaining ETL/ELT pipelines.
• Hands-on Power BI experience, including DAX, Power Query, data modeling, and dashboards.
• Familiarity with GitHub Actions and CI/CD pipelines.
• Experience working in Agile development environments.
• Strong SQL, data validation, debugging, and troubleshooting skills.
• Ability to develop, test, deploy, and maintain solutions independently.

Preferred Qualifications

• Tableau dashboard development experience.
• Experience with Databricks Delta Lake and Apache Spark.
• Familiarity with AWS Glue, IAM, CloudWatch, and other AWS data services.
• Experience with data warehousing, dimensional modeling, and data quality frameworks.
• Knowledge of automated testing and production monitoring.

I

"VForce Infotech is an Equal Opportunity Employer. We do not discriminate based on race, color, religion, sex, national origin, age, disability, veteran status, or any other legally protected status."