You meant it as a joke. They heard it as the truth. Both of those things are real.

Intent and impact both exist, and neither one cancels the other. You know what you meant, and they know how it felt from where they were sitting, and arguing about which is more valid just adds a second injury. Most of the damage in small moments comes from insisting only your half counts.

If something you said landed wrong, you can say both parts: you were joking, and it still landed badly, and you are sorry for the second one. That takes about eight seconds. Defending the joke takes much longer and fixes nothing.