Culture Secretary Lisa Nandy said: 'I think most people will accept that the very wealthiest shouldn't be in receipt of a free allowance'