You can be soft-spoken and still be the person whose “no” ends the conversation.

Volume and authority are not the same thing. What gives a refusal weight is that it does not move afterwards, and a quiet person who means it is far more final than a loud person who can be negotiated with. People work out very quickly which of those you are.

The next time you mean no, say it once, plainly, then stop talking. Do not stack three reasons on top of it, since reasons invite counterarguments. Hold the quiet that follows; it is uncomfortable for about four seconds, and then the matter is settled.