The on-loan Premier League star is devastated after his Derby County loan move was cut short