People living in this picturesque town are resisting calls to be governed by their larger neighbour.