You don’t owe loyalty to someone whose main hobby is hurting you.

Allegiance is a two-way arrangement, and it stops being that when only one party keeps the terms. Staying out of obligation to someone who repeatedly does you harm is not virtue but habit, often one that began as love and kept running after the reason left. Nothing about ending it requires you to hate them.

You are allowed to step back without a final confrontation or a full accounting. Reduce contact before you announce anything, because announcements invite negotiation. If you feel guilty, notice that the guilt shows how seriously you took the bond, not that you should stay.