At the Rise of AI Summit 2020, Adam Grzywaczewski, senior deep learning data scientist at NVIDIA, discusses the scaling laws that govern how increasing model and dataset size reliably improves model performance. According to the talk description, the session opens with a deep dive into this relationship between scale and performance, then covers the practical challenges of working with large models, including the costs associated with assembling and using large datasets. The talk concludes by connecting these scaling approaches to some of the biggest breakthroughs achieved in natural language processing and computer vision. The automatic transcript for this recording failed and is not usable, so this page is built only from the talk's title, chapter structure and YouTube description rather than the spoken content itself; no direct claims, numbers or examples from the talk can be reproduced here beyond what the description states.
Key takeaways
- Increasing model and dataset size is described as one of the most reliable ways to improve model performance.
- The talk covers scaling laws that govern the relationship between model/dataset size and performance.
- It addresses challenges tied to using large models, including the cost of large datasets.
- It connects scaling approaches to major breakthroughs in natural language processing.
- It connects scaling approaches to major breakthroughs in computer vision.
Questions answered in this talk
What are scaling laws in AI?
According to the description of Adam Grzywaczewski's talk at the Rise of AI Summit 2020, scaling laws describe how increasing model and dataset size reliably improves model performance.
What are the costs of using large AI models and datasets?
The talk description states that Grzywaczewski's session at Rise of AI Summit 2020 discusses challenges and costs associated with using large models and large datasets, though the transcript could not confirm specific figures.