The explanation can be brief when the choice belongs entirely to you.

Long justifications invite negotiation. When a decision is genuinely yours, the extra reasons you offer are not information, they are a bid for permission, and each one hands the other person somewhere to push. A short answer can feel rude at first, though mostly it is just unfamiliar. Explaining yourself and being granted consent are different activities.

Try answering in one sentence and then stopping, with something like I am not free, or I have decided against it. If someone asks for a reason you can give one, but notice how often the pause does the work on its own, and how rarely anyone actually asks.