When the whole group heads for trouble, being the one who turns around takes more nerve than going along.

Groups move on momentum, and the pull to stay with one is physical rather than reasoned. Leaving is hard not because anybody will punish you but because separating from a crowd feels like a mistake while you are doing it. Most people who go along knew, and went anyway.

Give yourself a plain exit line before you need it, something with no argument inside it. I am going to head off. Say it early rather than at the last possible moment, when it turns into a scene instead of a choice.