NYU Deep Learning Week 12 Deep Learning for Natural Language Processing (NLP)

Join the channel membership: https://www.youtube.com/c/AIPursuit/join

Subscribe to the channel: https://www.youtube.com/c/AIPursuit?sub_confirmation=1

Support and Donation: Paypal ⇢ https://paypal.me/tayhengee Patreon ⇢ https://www.patreon.com/hengee BTC ⇢ bc1q2r7eymlf20576alvcmryn28tgrvxqw5r30cmpu ETH ⇢ 0x58c4bD4244686F3b4e636EfeBD159258A5513744 Doge ⇢ DSGNbzuS1s6x81ZSbSHHV5uGDxJXePeyKy

Wanted to own BTC, ETH, or even Dogecoin? Kickstart your crypto portfolio with the largest crypto market Binance with my affiliate link: https://accounts.binance.com/en/register?ref=27700065

The video was published under the license of the Creative Commons Attribution license (reuse allowed). It is reposted for educational purposes and encourages involvement in the field of research. Source: https://youtu.be/6D4EWKJgNn0 Subscribe to Alfredo Canziani: https://www.youtube.com/channel/UCupQLyNchb9-2Z5lmUOIijw

0:00:00 – Week 12 – Lecture

LECTURE Part A: http://bit.ly/pDL-en-12-1 In this section we discuss the various architectures used in NLP applications, beginning with CNNs, RNNs, and eventually covering the state of-the art architecture, transformers. We then discuss the various modules that comprise transformers and how they make transformers advantageous for NLP tasks. Finally, we discuss tricks that allow transformers to be trained effectively. 0:00:44 – Introduction to deep learning in NLP and language models 0:13:48 – Transformer language model structure and intuition 0:32:55 – Some tricks and facts of Transformer Language Models and decoding Language Models

LECTURE Part B: http://bit.ly/pDL-en-12-2 In this section we introduce beam search as a middle ground between greedy decoding and exhaustive search. We consider the case of wanting to sample from the generative distribution (i.e. when generating text) and introduce “top-k” sampling. Subsequently, we introduce sequence to sequence models (with a transformer variant) and back-translation. We then introduce unsupervised learning approaches for learning embeddings and discuss word2vec, GPT, and BERT. 0:45:32 – Beam Search, Sampling and Text Generation 1:03:31 – Back-translation, word2vec and BERT's 1:22:43 – Pre-training for NLP and Next Steps ... https://www.youtube.com/watch?v=FVtmYHaoHJ4