Conservatives fear it could lead to more areas facing restrictions