They also worry about an increase in hate-motivated attacks and abuses