Understanding catastrophic forgetting for adaptive deep learning

Deep learning is still limited in practice tho it has progressed state of the art over the past few years. Current deep learning algorithms are rigid and static once trained and can't adapt to new data when deployed for inferencing. In this paper, we analyze the catastrophic forgetting phenomenon and show that the EWC algorithm for overcoming the same is not commutative for given tasks. And we also propose that learning to context switch between different sets of weights might overcome catastrophic forgetting.