An Abingdon town councillor is refusing to take his councillor allowance after the council approved a near 40 per cent increase.