People can be “dragged into narratives” about migration if politicians ignore genuinely-held concerns, deputy First Minister Emma Little-Pengelly has said.