The former presidents and first ladies had campaigned for Kamala Harris