East Durham College’s plans to build a new housing estate on an unused playing field have been refused over fears it will restrict access to sports facilities.