The Conservatives have described his advice to turn the heating on as 'cruel'