What you tolerate from your friends becomes part of your reputation too.

People do not read you alone. They read you in the group you stand in, and the things you let pass without comment become part of the picture. Silence looks like agreement from the outside, because from the outside there is nothing to tell them apart. That is not a rule about loyalty, only how anyone watching sorts a crowd into individuals.

Think of one thing your friends do that you would rather they did not. You do not need a speech. A short and unbothered comment in the moment is usually enough to move it.