The man was left scratching his head after his wife, who has been going through a difficult time in the last year, decided to seek solace by going for a meal with her male colleague