Several challenges are battering the farming sector, campaigners and MPs have warned.