A Scottish bakery, which had some famous fans, has closed down.