Councillors have approved a “site swap” for the RAAC affected Scalby School despite residents’ concerns.