Government borrowing costs have risen while the pound has dropped against the US dollar.