A problem in Model Fine Tuning where the model forgets previous training data. Often done when new data is to be fed in.

  • OpenAI bypasses this by feeding in all the old data + all the new data