A man has lost everything in a large scale fire in Abergavenny that has destroyed a piece of history, a Grade II listed building on Frogmore Street.