Poole Town fell to a disappointing 1-0 defeat at the BlackGold Stadium to Frome Town.