Residents south of Perth have called on the state government to ban visitors to a popular island to protect a species whose numbers have dwindled by 97 per cent.