The pub's previous owners said they had 'given up everything' to move there