The person smiling politely may be protecting the room rather than endorsing the remark.

A smile in an uncomfortable moment is not always agreement. Often it is the cheapest way to keep an evening from becoming an incident, chosen quickly by someone who has weighed the cost of objecting in front of everybody. Staying quiet there is a decision about the room rather than a verdict on the remark. Politeness can be a form of management.

If you noticed the remark too, find the person afterwards and say so quietly. That confirmation often matters more than a public stand would have. And if you were the one smiling, you are allowed to say something later rather than never.