A growing number of Liberal caucus members are calling for a secret ballot vote to decide whether Justin Trudeau should step down as party leader.