The detail dismissed as irrelevant may explain why the situation feels different to the other person.

What counts as relevant gets decided by whoever is doing the summarising, and they decide it from where they stand. A fact you set aside as beside the point, the timing, the phrasing, who else was in the room, may be the entire reason the same event lands differently for somebody else. Disagreements about facts are often disagreements about which facts are admitted. That dismissal happens early and quietly, before the argument begins.

When someone keeps returning to a detail you consider minor, ask why it matters to them instead of restating your main point. The answer usually explains the disagreement faster.