Some 50 residents headed out to challenge a homeowner newly arrived in the village