The singer will be starring in the hit musical for four weeks only.