The Prime Minister is “balancing the books on the backs of the poorest people in the world” in a bid to increase defence spending, a former international development secretary has suggested.