Historical allotments are at risk as residents say “get off our land”