"It wasn't right because I'm a married man."