Meta's Prometheus Supercluster Aims for 1 Million GPUs for AI Foundation Model Training

Meta has detailed its Prometheus supercluster in New Albany, Ohio, with plans to connect hundreds of thousands of GPUs in its first phase and ultimately target approximately 1 million GPUs for training its Muse foundation models. The company reported high pre-training and reinforcement learning 'goodput' rates.