Author Interview - Memory-assisted prompt editing to improve GPT-3 after deployment

#nlp #gpt3 #prompt

This is an interview with the authors of this work, Aman Madaan and Niket Tandon. Large language models such as GPT-3 have enabled many breakthroughs and new applications recently, but they come with an important downside: Training them is very expensive, and even fine-tuning is often difficult. This paper presents an adaptive method to improve performance of such models after deployment, without ever changing the model itself. This is done by maintaining a memory of interactions and then dynamically adapting new prompts by augmenting them with memory content. This has many applications, from non-intrusive fine-tuning to personalization.

OUTLINE: 0:00 - Intro 0:45 - Paper Overview 2:00 - What was your original motivation? 4:20 - There is an updated version of the paper! 9:00 - Have you studied this on real-world users? 12:10 - How does model size play into providing feedback? 14:10 - Can this be used for personalization? 16:30 - Discussing experimental results 17:45 - Can this be paired with recommender systems? 20:00 - What are obvious next steps to make the system more powerful? 23:15 - Clarifying the baseline methods 26:30 - Exploring cross-lingual customization 31:00 - Where did the idea for the clarification prompt come from? 33:05 - What did not work out during this project? 34:45 - What did you learn about interacting with large models? 37:30 - Final thoughts

Paper: https://arxiv.org/abs/2201.06009 Code & Data: https://github.com/madaan/memprompt

Abstract: Large LMs such as GPT-3 are powerful, but can commit mistakes that are obvious to humans. For example, GPT-3 would mistakenly interpret "What word is similar to good?" to mean a homonym, while the user intended a synonym. Our goal is to effectively correct such errors via user interactions with the system but without retraining, which will be prohibitively costly. We pair GPT-3 with a growing memory of recorded cases where the model misunderstood the user's intents, along with user feedback for clarification. Such a memory allows our system to produce enhanced prompts for any new query based on the user feedback for error correction on similar cases in the past. On four tasks (two lexical tasks, two advanced ethical reasoning tasks), we show how a (simulated) user can interactively teach a deployed GPT-3, substantially increasing its accuracy over the queries with different kinds of misunderstandings by the GPT-3. Our approach is a step towards the low-cost utility enhancement for very large pre-trained LMs. All the code and data is available at this https URL.

Authors: Aman Madaan, Niket Tandon, Peter Clark, Yiming Yang

Links: Merch: http://store.ykilcher.com TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannick ... https://www.youtube.com/watch?v=3ks2gpqAKY8