Many will be left without support preventing them from becoming further displaced, non-profits warn