Skip to main content

LLM Pre-training & Distributed Engineer (AI Infrastructure)

Expired
This role has expired and is no longer accepting applications. Browse similar roles →
Hyphen Connect Limited
Australia
remote
Full Time / Permanent

Apply for this job

Posted 3 months ago
This role is expired

These roles are hiring now

View all similar roles →

Group Owner Cloud AI Infrastructure

Telstra
Australia
remote
  • Lead AI infrastructure strategy, roadmap and operations across Telstra
  • Senior leadership experience in large-scale AI infrastructure, cloud, platform
  • GPU platforms, model serving, DevSecOps, Infrastructure as Code, SRE
Posted 1 month ago

Storage Engineer, AI Cluster Commissioning

Firmus Technologies
Melbourne, VIC
  • Deploy, configure and troubleshoot high-performance storage for AI clusters
  • 5+ years experience with enterprise, HPC, cloud or AI storage infrastructure
  • VAST Data, Weka, IBM Storage Scale, PowerScale, Ceph, Lustre, Linux
Posted 6d ago

Senior AI Infrastructure Engineer, Kubernetes

Firmus Technologies
Sydney, NSW
  • Design & operate GPU-accelerated Kubernetes infrastructure at AI-factory scale
  • 7+ years platform engineering, 3+ years at principal-level
  • Kubernetes internals, Go/Rust, bare-metal, GPU/NVIDIA operators, CNI
Posted 6d ago

Network Engineer, AI Cluster Commissioning

Firmus Technologies
Melbourne, VIC
  • Deploy and commission high-performance AI networks
  • 5+ years network engineering experience
  • Ethernet, RDMA, RoCE, BGP, Linux, network automation
Posted 6d ago

We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.

Responsibilities:

  • Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
  • Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
  • Automate checkpointing and failure recovery during month-long training runs.

Required Skills:

  • Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
  • Experience managing SLURM or Kubernetes-based GPU clusters.
  • Strong systems engineering background (C++, CUDA, Python).