The outgoing Conservative Party leader was making his final appearance at the despatch box before stepping down.