No one returns with good-will to the place which has done him a mischief.

Places hold what happened in them. A house, an office or a town where you were badly treated keeps that flavour long after the people involved have gone, and going back is rarely as neutral as you expect. Requiring yourself to feel nothing there is asking for something you cannot supply.

If you have to return somewhere that went badly, plan for the feeling instead of being ambushed by it. Keep the visit short, take someone with you, decide beforehand what you will do afterwards. And if you do not have to go back, you are allowed not to.