The Reform UK member previously served as a prisons minister under the Conservative administration of Sir John Major.