Jobiglo

No results.

Deep Learning Team Lead - LLM Infrastructure

AI21 Labs · Tel Aviv-Yafo

New
Senior 🇬🇧 English
Python PyTorch FSDP Megatron DeepSpeed vLLM TensorRT-LLM Slurm Kubernetes

Job description

About the role

AI21, a company that has trained multi-billion parameter Large Language Models, is looking for a Deep Learning Engineer – Team Lead. You will lead a team of engineers working on LLM training and inference infrastructure, collaborating closely with applied researchers to develop and scale new training methods, improve performance, optimize model serving and make the best use of advanced hardware accelerators. This is a hands-on technical leadership role combining engineering, research and team management.

Key responsibilities

  • Lead, mentor and manage a team of deep learning engineers, including task delegation, technical guidance and project execution.
  • Develop, improve and maintain the LLM training framework and supporting infrastructure.
  • Optimize training performance, scalability and model quality through parallelism techniques, training SDK improvements and new model architectures.
  • Research, develop, test and integrate new training methods and reinforcement learning environments.
  • Adapt training infrastructure to the latest GPU hardware and accelerators.
  • Optimize model serving for high throughput and low latency in production and training environments.
  • Work with applied researchers on critical post-training efforts.
  • Design, run and analyze training and evaluation experiments to improve LLM performance across tasks.
  • Debug complex issues across the training and inference stack, from model behavior to distributed systems and hardware performance.

Required profile

  • 3+ years of experience developing production software, with strong Python skills and solid engineering fundamentals (testing, debugging, performance optimization, unfamiliar codebases).
  • Proven experience leading and managing engineering teams, including mentoring, delegation and project execution.
  • BSc or higher in Computer Science, Electrical Engineering or a related field.
  • Nice to have: experience with distributed, containerized GPU training environments such as Slurm or Kubernetes, and launching/debugging multi-node training jobs.

Required skills

  • Hands-on understanding of end-to-end LLM training in PyTorch, including data, model, tensor and pipeline parallelism, mixed precision and memory/compute tradeoffs.
  • Understanding of distributed training frameworks such as FSDP, Megatron or DeepSpeed; specific framework experience is not required.
  • Deep understanding of LLM architectures, including attention variants, MoE, positional encodings and KV cache.
  • Ability to debug issues such as loss spikes, NCCL hangs, log-probability mismatches and inefficient kernels.
  • Nice to have: distributed training (FSDP, Megatron, DeepSpeed, JAX), post-training and RL (SFT, DPO, GRPO, verl, TRL), inference systems (vLLM, SGLang, TensorRT-LLM) and model efficiency (quantization, distillation, speculative decoding).

Questions fréquentes

Le salaire n'est pas communiqué publiquement par le recruteur. Vous pouvez postuler et négocier directement avec AI21 Labs.
Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.

Why are you reporting this job?

Thank you for your report. We will review this job.

Explore further

Salaries, guides and searches in Israel.

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

Published 7 hours ago

Expires 1 month from now

5 views · 0 interested

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...

AI21 Labs

Tel Aviv-Yafo