‘He wasn’t what he was on the telly,’ actor said