Activists urge govt to take steps to restore the 350-year-old heritage structure to its former glory at the earliest