'The town is in dire need of affordable homes and social housing'