Father claims they were ‘naively or stupidly’ not aware of any permits that might have been required to build the track