It had left residents feeling unsafe