The owner of the quarry claimed residents’ HGV traffic concerns were not caused by his lorries