Question-Answering in NLP (Extractive QA and Abstractive QA)

Search is a crucial functionality in many applications and companies globally. Whether in manufacturing, finance, healthcare, or almost any other industry, organizations have vast internal information and document repositories.

Unfortunately, the scale of many companies' data means that the organization and accessibility of information can become incredibly inefficient. The problem is exacerbated for language-based information. Language is a tool for people to communicate often abstract ideas and concepts. Naturally, ideas and concepts are harder for a computer to comprehend and store in a meaningful way.

How do we minimize this problem? The answer lies with semantic search, specifically with the question-answering (QA) flavor of semantic search.

This article will introduce the different forms of QA, the components of these 'QA stacks', and where we might use them.

🌲 Pinecone article: https://www.pinecone.io/learn/question-answering/

🤖 70% Discount on the NLP With Transformers in Python course: https://bit.ly/3DFvvY5

🎉 Subscribe for Article and Video Updates! https://jamescalam.medium.com/subscribe https://medium.com/@jamescalam/membership

👾 Discord: https://discord.gg/c5QtDB9RAP

00:00 Meaningful Search 01:23 Use-case 02:22 Open Domain QA (ODQA) 06:41 SQuAD Format 10:45 Quick Preprocessing 15:18 Creating Context Vectors Database 23:24 Open-book Extractive QA 32:50 Open-book Abstractive QA 41:53 Closed-book Abstractive QA 47:27 Final Thoughts ... https://www.youtube.com/watch?v=-td57YvJdHc