In this webinar, we will review the basic steps needed to enable DeepSpeed on Gaudi and show how the ZeRO1 and ZeRO2 memory optimizers and Activation Checkpointing are used to reduce memory usage on a large model. We’ll also show you how to use Gaudi APIs to detect the peak memory of any model and provide guidance on when to use these techniques. A live Q&A to follow.
PM @ Habana Labs, Ex Meta, PayPal, Conversation.one
Loves ML, DL, Gardening and Board Games
Community support is provided during standard business hours (Monday to Friday 7AM - 5PM PST). Other contact methods are available here.
Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.