A bid to give Swindon's Abbey Stadium a small degree of protection from redevelopment, particularly into housing, has been turned down.