Senior Machine Learning Systems Engineer – LLM Inference Performance on AWS Trainium

Company Description SARACA is a global engineering R&D services company partnering with more than 25 Fortune 500 customers across industries including MedTech, Aerospace, Rail, Automotive, Semiconductor, Defense, and Farm equipment. The organization is ISO 13485 certified and has deep expertise in medical device design and development, covering embedded software, UI/UX, mechanical, systems, and product testing. SARACA’s leadership and technical teams bring extensive experience in product engineering services and specialized knowledge of IEC 62304, EU MDR, and scientific and regulatory documentation in MedTech. With a team of 400+ engineers and management consultants working on onsite and offsite projects worldwide, SARACA focuses on solving complex business problems and accelerating customers’ growth. SARACA is an equal-opportunity employer fostering a culture of innovation, continuous learning, and industry leadership.

Job Description

We are seeking a hands-on Machine Learning Systems Engineer to optimize production-grade LLM inference workloads across current and next-generation AWS Trainium platforms, including Trainium2 and Trainium3.

The role focuses on improving decode latency, inference throughput, price-performance, and long-context serving capabilities while ensuring model correctness and reproducible benchmarking. The engineer will work with a senior engineering team to optimize LLM architectures, implement advanced inference techniques, and benchmark performance against leading GPU platforms.

This is a highly technical, customer-facing role requiring end-to-end ownership of optimization, performance analysis, technical documentation, and customer enablement.

Key Responsibilities

• Optimize LLM inference performance on AWS Trainium platforms, focusing on decode latency, throughput, and serving cost.
• Implement and evaluate speculative decoding and native Multi-Token Prediction (MTP) for Qwen-family models using vLLM on AWS Neuron.
• Design and execute reproducible performance benchmarks comparing AWS Trainium with leading GPU platforms.
• Analyze latency-throughput trade-offs, price-performance, concurrency, cache behavior, and workload-specific service-level objectives.
• Optimize inference workloads across Trainium generations, including Trainium2 and Trainium3.
• Scale LLM serving across multi-chip Trainium systems, particularly for long-context workloads involving 100K+ tokens.
• Extend inference support to additional model architectures, including hybrid state-space models and Mixture-of-Experts (MoE) models.
• Profile and optimize accelerator workloads at the framework, compiler, and kernel levels.
• Validate model output correctness and quality before accepting performance improvements.
• Develop technical documentation, benchmark reports, implementation guides, and operational runbooks.
• Collaborate with customer engineering teams and leadership to communicate technical findings, optimization strategies, and business impact.
• Conduct technical workshops and enable customers to independently reproduce benchmarks and optimize their inference environments.

Required Qualifications

• 5+ years of experience in machine learning or systems engineering, including at least 2 years focused on production ML inference or model serving.
• Strong programming skills in Python and PyTorch, including the ability to read and modify model implementations.
• Hands-on experience with LLM serving frameworks such as vLLM, TensorRT-LLM, SGLang, or Triton Inference Server.
• Strong understanding of LLM inference concepts, including prefill, decode, continuous batching, KV-cache management, tensor parallelism, and expert parallelism.
• Experience profiling and optimizing accelerator workloads at the framework, compiler, or kernel level.
• Strong understanding of performance benchmarking methodologies, latency-throughput analysis, and inference cost optimization.
• Ability to analyze attention mechanisms, Mixture-of-Experts routing, and model execution behavior.
• Excellent written and verbal communication skills, with the ability to explain technical trade-offs to engineering teams and business stakeholders.

Preferred Qualifications

• Hands-on experience with AWS Trainium, AWS Inferentia, AWS Neuron SDK, NxD Inference, vLLM on Neuron, or Neuron Kernel Interface (NKI).
• Experience migrating and optimizing workloads across accelerator generations.
• Experience implementing or evaluating speculative decoding techniques, including draft-model approaches, Medusa, EAGLE, or Multi-Token Prediction (MTP).
• Kernel development experience using NKI, Triton, or CUDA.
• Experience with hybrid state-space architectures such as Mamba or large-scale Mixture-of-Experts models.
• Familiarity with FP8/INT8 quantization and model accuracy validation.
• Experience in customer-facing engineering, technical consulting, or customer enablement.

Key Success Factors

• Demonstrated improvements in LLM inference latency, throughput, and serving cost without compromising model quality.
• Ability to deliver accurate, reproducible, and defensible performance benchmarks.
• Experience optimizing workloads representative of real-world production environments.
• Strong ownership of implementation, performance analysis, documentation, and customer communication.
• Ability to enable customer teams to independently operate, benchmark, and optimize the inference stack.
• Interested candidates can share their updated resume along with relevant experience in LLM inference optimization, vLLM, PyTorch, and AWS Trainium/Neuron.