Villagers have said they are considering moving out of Felton as the issue is “only getting worse”