Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 19 implied HN points • 22 Feb 24
- Catastrophic forgetting happens when language models forget things they learned before as they learn new information. It's like a student who forgets old lessons when they study new subjects.
- Language models can change their performance over time, sometimes getting worse instead of better. This means they can produce different answers for the same question at different times.
- Continuous training can make models forget important knowledge, especially in understanding complex topics. Researchers suggest that special training techniques might help reduce this forgetting.