Rob Holt claims he was ordered to leave The Wickham, in Brisbane , on Sunday after a staff member told him people might be offended by the cap.