A federal minister was allegedly biased after refusing to extend the lease for an undeveloped uranium mine due to a "promise" made to the traditional owners.