Former Labor treasurer Wayne Swan has been forced to defend his blistering attack on the Reserve Bank.