A former Premier League referee has let a startling admission slip.