GAN BERT: Generative Adversarial Learning for Robust Text Classification (Paper Explained) #GANBERT

GAN is a powerful way to generate data more pseudo data to train models. This paper shows how to integrate semi-supervised GAN with the most popular NLP pre-trained model, BERT.

0:00 - Intro 2:19 - Semi-supervised GANs 4:00 - Discriminator Loss Function 6:24 - Generator Loss Function 9:06 - GAN-BERT 11:59 - Unlabeled Real Examples 13:36 - Experiments 17:32 - Takeaways

GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples

Paper: https://www.aclweb.org/anthology/2020.acl-main.191/

Abstract Recent Transformer-based architectures, e.g., BERT, provide impressive results in many Nat- ural Language Processing tasks. However, most of the adopted benchmarks are made of (sometimes hundreds of) thousands of exam- ples. In many real scenarios, obtaining high- quality annotated data is expensive and time- consuming; in contrast, unlabeled examples characterizing the target task can be, in gen- eral, easily collected. One promising method to enable semi-supervised learning has been proposed in image processing, based on Semi- Supervised Generative Adversarial Networks. In this paper, we propose GAN-BERT that ex- tends the fine-tuning of BERT-like architec- tures with unlabeled data in a generative adver- sarial setting. Experimental results show that the requirement for annotated examples can be drastically reduced (up to only 50-100 an- notated examples), still obtaining good perfor- mances in several sentence classification tasks.

Code: https://github.com/crux82/ganbert ... https://www.youtube.com/watch?v=vAQsGi6NctY