The Frenchman and Kyle Walker were seen begging the referee to reconsider