The question of when — or even if — a border poll should be held continues to split voters.