Ahead of the final budget outcome later this month, the Treasurer confirmed Australia has improved its debt position, saving taxpayers on interest fees.