The former Saturdays singer was called "the worst wife and mother" after going on a child-free holiday