Police hold concerns over what access an ex-priest has had to children after he was arrested on the streets of a major city amid an ongoing police investigation.