Adultery always begins with the adulterer(s) claiming to themselves and to others that the relationship is "harmless" because it hasn't crossed a certain line. The line where it becomes wrong is the line where you start having to rationalize like that.