Be someone whose absence does not make people question what you said in their presence.

How you speak about people when they are absent teaches everyone present what you say about them. Nobody has to be told; they simply extrapolate, and reasonably so. The person who criticises freely in private conversation is understood to do the same in the other direction.

A workable rule is to say nothing about someone you would not repeat to them. It is restrictive at first and it makes people trust you in a way that is hard to build any other way. Try it for a fortnight and notice both what you stop saying and how conversations change.