One driver said he felt helpless and frustrated after getting stuck for hours with his partner and their one-year-old baby