How to make your CPU as fast as a GPU - Advances in Sparsity w/ Nir Shavit

#ai #sparsity #gpu

Sparsity is awesome, but only recently has it become possible to properly handle sparse models at good performance. Neural Magic does exactly this, using a plain CPU. No specialized hardware needed, just clever algorithms for pruning and forward-propagation of neural networks. Nir Shavit and I talk about how this is possible, what it means in terms of applications, and why sparsity should play a much larger role in the Deep Learning community.

Sponsor: AssemblyAI Link: https://www.assemblyai.com/?utm_source=youtube&utm_medium=social&utm_campaign=yannic_autochapters

Check out Neural Magic: https://neuralmagic.com/ and DeepSparse: https://github.com/neuralmagic/deepsparse

OUTLINE: 0:00 Introduction 1:08 Sponsor: AssemblyAI 2:50 Start of Interview 4:15 How the NIR company was founded? 5:10 What is Sparsity about? 9:30 Link between the human brain and sparsity 12:10 Where should the extra resource that the human brain doesn't have go? 14:40 Analogy for Sparse Architecture 16:48 Possible future for Sparse Architecture as standard architure for Neural Networks 20:08 Pruning & Sparsification 22:57 What keeps us from building sparse models? 25:34 Why are GPUs so unsuited for sparse models? 28:47 CPU and GPU in connection with memory 30:14 What Neural Magic does? 32:54 How do you deal with overlaps in tensor columns? 33:41 The best type of sparsity to execute tons of CPU 37:24 What kind of architecture would make the best use out of a combined system of CPUs and GPUs? 41:04 Graph Neural Networks in connection to sparsity 43:04 Intrinsic connection between the Sparsification of Neural Networks, Non Layer-Wise Computation, Blockchain Technology, Smart Contracts and Distributed Computing 45:23 Neural Magic's target audience 48:16 Is there a type of model where it works particularly well and the type where it doesn't?

Links: Homepage: https://ykilcher.com Merch: https://ykilcher.com/merch YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ykilcher.com/discord LinkedIn: https://www.linkedin.com/in/ykilcher

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://www.subscribestar.com/yannickilcher Patreon: https://www.patreon.com/yannickilcher Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n ... https://www.youtube.com/watch?v=0PAiQ1jTN5k