One Trillion Parameters and Beyond - Lessons Learned from Training Extremely Large Language Models on AWS
Speakers: Emily Webber, Machine Learning Specialist Solutions Architect, AWS | Neel Kishan, Business Development Manager AI/ML, AWS
GPT-3, Wu Dao, Megatron-LM, Compositional Transformers and more. In this session we'll explore a variety of extreme scale NLP models and the distributed science and architecture that make them tick. In this session you'll learn about the newly launched distributed training toolkits on the AWS cloud and learn about how customers from Hyundai to Hugging Face are pushing the boundaries on transformer-based models. You'll learn about custom hardware AWS is developing to lower costs and boost performance. You'll also learn about how to partner with AWS to bring your unique technology to the enterprise cloud market!
