Hello. Are you quite sure that history of optimizer (e.g. moments) should be zeroed at the beginning of each epoch?
Hello. Are you quite sure that history of optimizer (e.g. moments) should be zeroed at the beginning of each epoch?