DA

Senior AI Research Engineer - LLM Training & Inference Infrastructure

datadog· 68 open roles

Paris, Germany On-siteFull-time1 weeks ago
Salaryest.
€80,000 - €120,000
Experience
Mid
Job Type
Full-time
Posted
1 weeks ago
Apply Now

Boost your chances at datadog

Tailor your resume to this exact job and generate a matching cover letter in about 60 seconds with JobEase — our AI application assistant.

  • ATS-optimized resume
  • Personalized cover letter
  • Match score & keyword gaps
Tailor my application

Free to start · no card required

Get more other jobs in your inbox

Verified daily — no ghost listings.

About This Role

Senior AI Research Engineer builds and operates distributed training and inference infrastructure for large language models at Datadog. The role partners with research scientists to design systems that schedule GPU workloads, optimize memory, and ensure reliable long training runs. It also focuses on improving researcher experience and maintaining high-performance execution environments for RL and inference.

Key ResponsibilitiesAI-extracted

  • 1
    Design, build, and operate distributed training and inference infrastructure for large language models.
  • 2
    Scale training with data, tensor, pipeline, and expert parallelism, and implement memory optimization and checkpointing.
  • 3
    Develop inference and rollout systems that support evaluation, experimentation, and reinforcement learning without impacting training.
  • 4
    Create execution environments and sandboxes for online reinforcement learning and agent post‑training.
  • 5
    Own continuous performance validation, including benchmarks of rollout speed, training throughput, and learning behavior.
  • 6
    Improve researcher experience by enhancing job submission, scheduling, quotas, GPU utilization, and observability.
  • 7
    Collaborate with platform teams on cluster capacity, networking, and scheduling across GPU providers.
  • 8
    Investigate performance and reliability issues across hardware, software, and distributed systems.

Requirements

  • You have depth in distributed systems and ML infrastructure for training and inference at scale; experience with Ray, Kubernetes, Slurm, or similar frameworks is a plus
  • You are proficient in Python and in at least one systems language (e.g., Rust, C++, or Go) ● You have experience operating large-scale training or serving infrastructure, or schedulers and orchestration systems for GPU or other high-performance workloads
  • You understand what limits throughput/latency in distributed inference workloads: communication, memory, scheduling, and stragglers
  • You have strong debugging, profiling, and incident-diagnosis skills, ideally including on-call experience
  • You can explain design and performance trade-offs clearly to both technical and non-technical audiences
  • Prior experience training machine learning models is valuable but not essential for engineers with strong distributed systems or HPC backgrounds
  • You are familiar with distributed training frameworks such as Megatron-LM, TorchTitan, or DeepSpeed
  • You have experience with LLM inference engines such as vLLM or SGLang, including batching and KV-cache management
  • You have experience with RL training systems and rollout infrastructure
  • You have experience with collective communication (NCCL), RDMA, or low-latency networking
  • You have experience with GPU benchmarking, profiling, and cluster performance monitoring
  • You have experience building sandboxed code execution or large-scale evaluation infrastructure
  • You have contributed to open-source ML or distributed systems projects

Perks & Benefits

Competitive global benefits
New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
Opportunity to collaborate closely with colleagues across the Datadog offices in New York City and Paris
Opportunity to attend and present at conferences and meetups
Intra-departmental mentor and buddy program for in-house networking
An inclusive company culture, ability to join our Community Guilds (Datadog employee resource groups)
Benefits and Growth listed above may vary based on the country of your employment and the nature of your employment with Datadog.

Apply to This Job in Minutes

Generate ATS-optimized resume + cover letter + interview prep with Jobease.ca AI. Complete your application faster.

Get Started Free

Similar Jobs

RE

Analytics Engineer

redditRemote
View
RE

Associate General Counsel, Ads Product

redditRemote
View
RE

Group Product Manager, Finance Technology

redditRemote
View