A goal after just 80 seconds was enough to send Stourbridge crashing out of the FA Trophy.