Sir Jim Walker, who grew the brand over six decades, has died, aged 80.