Peter Costello has taken aim at Daniel Andrews during a Sydney conference telling 700 people his Victorian government had been the “worst in Australia”.