The higher you climb, the further you have to fall, and Labor has always known it will lose ground this weekend.