A controversial proposal to use the former St Mary's College in Blackburn owned by the Issa brothers as an events venue has been dropped.