More than 40 'affordable rental homes' are to be built on the 'poorly used' recreational land