Victorian Opposition Leader John Pessuto is refusing to stand down from his position despite mounting pressure and whispers of a mutiny.