Some fear the racing meets post a 'serious danger' to others on the roads