Production Systems Engineer, AI Systems

Meta is seeking a Systems Engineer to join our Release to Production (RTP) team working on AI/ML initiatives. Our servers and data centers are the foundation upon which our rapidly scaling infrastructure operates efficiently to deliver our innovative services. The RTP team is responsible for the end-to-end Hardware Lifecycle of all Meta servers including prototyping of experimental HW, pre-production hands-on system and hardware debugging and stress testing, enabling production-ready system monitoring, automated provisioning and automated remediation of issues. RTP team also helps in exploring, developing and productizing high-performance software and hardware technologies for AI at datacenter scale.RTP Engineers have a large swath of XFN partners they work closely with e.g. HW/SW co-design teams, hardware designers, networking teams, system manufacturers, component vendors, capacity engineering, production engineering, production services, and data center operations teams to enable new systems that will be deployed in our production data centers.

Production Systems Engineer, AI Systems Responsibilities:

• GPU Subject Matter Expert (SME) with knowledge of hardware technology/collective communications helping drive GPU validation across AI Platforms as well as large cluster initiatives
• Contribute to new feature/technology development/validation across hardware/software stack
• Drive hacks for future technology explorations in AI space
• Leverage deep understanding of HW/SW systems to execute full product life-cycles (prototyping, deployment, and support)
• Troubleshoot, diagnose and root cause of system failures and isolate the components/failure scenarios while working with internal & external stakeholders
• Champion engineering and operational excellence, establishing metrics and process for regular assessment and improvement
• Develop visibility through data visualization and implement systemic solutions to hardware health issues
• Proactively create experiments and tooling to detect and diagnose hardware/firmware/software health issues

Minimum Qualifications:

• Bachelor’s degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience.
• 10+ years of work experience in one or more domains such as: ASIC development (Silicon design, bringup/characterization, deployment), compute (ARM, x86), AI-ML hardware/software (GPUs, TPUs, Communication Libraries), networking (e.g. Switches, NICs), server architecture
• Knowledge of server architecture and components across Compute/Storage/AI Systems
• Experience troubleshooting problems at system level, crossing across multiple components, as well as hardware/ software boundaries
• 5+ years of experience supporting hardware design teams (silicon design or datacenter platform design: compute, storage, etc.) in end-to-end product development cycles
• 5+ years of experience in high-impact/high profile/at scale project leadership and ownership

Preferred Qualifications:

• 5+ years of experience with one of the following AI systems spaces: Accelerator (GPU/ASIC) design/bring up, Kernel development, Performance optimization (e.g., NVIDIA, AMD, Intel, or other misc accelerator), communication libraries (e.g., NCCL, RCCL, MPI), performance enablement, tracing, profiling and debugging
• Experience with architecture of disaggregated systems at scale
• Experience supporting internal/external customers running large scale AI Training jobs

About Meta:

Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.