Viewers took to X to share their disappointment