AWS Migration Platform Engineer

Role OverviewWe are seeking an experienced AWS Migration Platform Engineer to design, develop, and operate a secure, scalable, and highly reliable migration platform for transferring large volumes of object data from OBSv2 to Amazon S3.The role focuses on building automated migration workflows, orchestrating AWS services, managing high-volume data transfers, and implementing robust governance, monitoring, security, and operational controls. The engineer will be responsible for developing production-ready migration capabilities that support both manifest-based and dynamic migration approaches across enterprise environments.The ideal candidate will have strong hands-on experience with AWS serverless services, containerized workloads, Python, DynamoDB, rclone, networking, and large-scale object storage migrations.Key Responsibilities

• Design, develop, and maintain end-to-end migration workflows to transfer object data from OBSv2 source buckets to Amazon S3 destination buckets.
• Build and operate AWS-based orchestration solutions using Step Functions, Lambda, ECS Fargate, AWS Batch, DynamoDB, SQS, and rclone.
• Implement manifest-based and dynamic migration strategies, including object discovery, prefix-based partitioning, batching, sharding, task sizing, retries, and recovery mechanisms.
• Develop centralized governance capabilities for bandwidth allocation, queue management, job prioritization, throttling, and migration progress tracking.
• Build reusable, containerized rclone migration components supporting high-volume data transfers, checksum validation, dynamic bandwidth control, and progress reporting.
• Design DynamoDB-backed state management for migration jobs, tasks, queues, bandwidth allocation, and completion summaries.
• Implement comprehensive observability using Amazon CloudWatch dashboards, logs, metrics, alarms, health checks, and operational reports.
• Configure secure multi-account AWS deployments using IAM roles, cross-account access, STS assume-role, Secrets Manager, and least-privilege security principles.
• Optimize migration performance by balancing object volumes, transfer concurrency, compute capacity, network throughput, and bandwidth constraints.
• Collaborate with infrastructure, security, networking, and data engineering teams to resolve dependencies and ensure migration readiness.
• Develop CI/CD pipelines, deployment automation, environment-specific configurations, runbooks, and operational documentation.
• Support integration testing, performance testing, production cutover, issue resolution, restart procedures, and post-migration validation.
• Ensure reliable production operations through proactive monitoring, failure handling, recovery planning, and stakeholder reporting. Mandatory Technical Skills
• AWS Serverless and Orchestration
• Strong hands-on experience with AWS Lambda, Step Functions, SQS, SNS, EventBridge, and API Gateway.
• Experience with AWS Systems Manager (SSM) Parameter Store and Secrets Manager.
• Ability to design fault-tolerant, event-driven workflows with retry and error-handling mechanisms.
• AWS Compute and Containers
• Strong working knowledge of Amazon ECS Fargate and AWS Batch.
• Experience with container images, task definitions, sidecar patterns, resource sizing, and runtime configuration.
• Ability to optimize containerized workloads for large-scale data transfer operations.
• Amazon S3 and Object Storage
• Deep understanding of Amazon S3, S3 Intelligent-Tiering, S3-compatible APIs, object metadata, prefixes, and lifecycle management.
• Experience handling large object transfers, high object counts, and enterprise-scale storage environments.
• Understanding of S3 storage optimization and destination validation.
• Object Storage Migration and rclone
• Hands-on experience migrating large object datasets from on-premises or S3-compatible object storage platforms to Amazon S3.
• Experience with Dell ECS, OBSv2, or similar enterprise object storage systems is highly desirable.
• Strong experience configuring and operating rclone for high-volume transfers, including concurrency, transfers, checkers, retries, checksum validation, no-traverse, statistics, logging, and bandwidth limiting.
• Python Development
• Strong Python programming skills for developing AWS Lambda functions, Fargate services, and migration automation utilities.
• Experience with boto3, JSON processing, exception handling, structured logging, dataclasses, streaming files, and API integrations.
• Ability to develop reusable, maintainable, and production-ready automation components.
• DynamoDB and State Management
• Experience designing and operating DynamoDB tables, partition and sort keys, indexes, updates, scans, and batch writes.
• Understanding of Decimal handling, job tracking, task orchestration, queue state, and distributed workflow state management.
• AWS Networking
• Strong understanding of AWS Direct Connect, Transit Gateway, VPCs, private subnets, routing, S3 Gateway Endpoints, and NAT Gateway configurations.
• Experience designing high-throughput network paths for large-scale data transfers.
• Ability to troubleshoot network connectivity, routing, and performance bottlenecks.
• IAM and Security
• Strong understanding of IAM policies, roles, cross-account access, STS assume-role, API keys, and Secrets Manager.
• Experience implementing least-privilege access, secure credential management, and privacy safeguards.
• Ability to support secure deployment and operations across multiple AWS accounts.
• Monitoring and Observability
• Hands-on experience with CloudWatch metrics, logs, alarms, dashboards, and structured logging.
• Ability to implement operational telemetry, migration progress reporting, health checks, and failure alerts.
• Experience developing monitoring and troubleshooting procedures for production workloads.
• CI/CD and Deployment Automation
• Experience with GitLab repositories, CI/CD pipelines, and automated infrastructure deployment.
• Ability to manage environment-specific configuration, deployment validation, release controls, and safe resource teardown.
• Familiarity with version control, code reviews, and production release practices. Domain Knowledge and Required Expertise
• Understanding of large-scale object storage migration patterns, including manifest-based migration, dynamic object discovery, prefix partitioning, batching, and validation.
• Ability to balance migration throughput against data volume, object count, compute resources, network capacity, and rclone concurrency.
• Knowledge of bandwidth allocation, dynamic throttling, queue processing, job prioritization, retries, and escalation mechanisms.
• Strong operational discipline covering monitoring, runbooks, restart and recovery procedures, completion checks, and stakeholder communications.
• Ability to create clear technical documentation, architecture diagrams, input/output contracts, implementation notes, and support procedures.
• Experience supporting enterprise-scale migration programs in production environments, preferably with security and compliance requirements. Expected Deliverables
• A production-ready migration orchestration platform supporting both manifest-based and dynamic migration workflows.
• Centralized governance for bandwidth allocation, queue processing, throttling, job prioritization, and completion reporting.
• A reusable rclone-based migration runtime with progress reporting, checksum validation, and dynamic bandwidth control.
• DynamoDB-backed state management for migration jobs, tasks, queues, bandwidth allocation, and execution summaries.
• CloudWatch dashboards, alarms, logs, health checks, and operational telemetry.
• Automated deployment pipelines, environment-specific configurations, operational runbooks, and technical documentation.
• A secure, scalable, and recoverable platform ready for enterprise production migration and cutover. Preferred Qualifications
• Experience with Dell ECS, OBSv2, or other S3-compatible enterprise object storage platforms.
• Experience delivering large-scale enterprise data migrations in regulated or business-critical production environments.
• Experience with hub-and-spoke architecture and centralized orchestration across multiple AWS accounts.
• AWS certifications such as AWS Certified Solutions Architect or AWS Certified Developer.
• Experience with infrastructure as code and automated cloud resource provisioning.
• Strong technical communication skills and the ability to provide clear, leadership-ready status reports.