‘It’s very condescending,’ the actor said