Hey there! Hard to say, but there is definately a problem…
I assume the encoder is working well?
The fact that it acts differently in one direction to another makes me think there can be a problem with the electrical zero… either it is not being found right, or it is losing the absolute zero position, for example due to missed encoder pulses…