Opinion is split over a major plan to redevelop two Oxford colleges' sports facilities.