A fire broke out in a cloth factory in Ahmedabad on Sunday afternoon.