Some even called the former Jame Bond star’s his lilt “offensive”