Local campaigners fear Labour's housing reforms could change the character of their idyllic home forever.