The adult who joins the play should not always become the person in charge of it.

Adults tend to arrive in a game and start improving it. The rules get clarified, the turns get organized, the direction gets set, and within a minute the play has become a supervised activity with an authority in it. Children usually go along, because that is what happens when a grown-up joins, and what they lose is the part where they were deciding.

Join the next game as an ordinary player and ask what your character is supposed to do. Follow their rules even when they make no sense. Save the organizing for when someone actually asks for it.