Rising house prices and a lack of community driven by an influx of tourists propmpted the call for a referendum