More than 30 people are objecting to redevelopment plans for two Oxford colleges' sports facilities.