This sounds like one of those situations in which no good deed goes unpunished.  Your primary legal protection should already be in place, however, if the founders of your association included standard individual protections in your governing documents.  Your governing documents ought to limit your personal liability only to those situations that would involve your personal self-dealing or intentional misconduct (which I assume is not the case) and ought to include good indemnification language to protect you if you do get sued.  (See Ready Reference Page: “Bylaws Function as ‘Constitution’ of Nonprofit Corporations”)  The association should also have directors and officers insurance in place to provide the resources to protect you if you get sued.  Even if your decisions were wrong, you should not be personally liable if they were made in good faith and were what you believed to be in the best interests of the association. Careful preplanning could have reduced your risk considerably.

As a practical matter, you ought to cooperate with them as much as possible in the transition and help them find replacement members of the board to carry on when you leave.  Physicians have learned that showing concern for patients is one of the best ways to avoid malpractice litigation and those lessons should apply here. Some people are just jerks, but even they can realize eventually that litigation is seldom the answer in a situation like this.