This is a matter of state law and may not be uniform throughout the country.  Your state nonprofit corporation law or case law may provide an answer, but it is likely that there is no specific statute or case on the question.

If you have failed to give the notice required by your bylaws (30 days notice is more than is likely to be required by a statute), any action taken at the meeting could be found to be illegal and ineffective.  Your safest course of action would be to cancel the improperly noticed meeting and set a new one with the proper notice.

Assuming you are not subject to a state open meeting law, as a practical matter, that may not be necessary, however, depending on your tolerance for risk.  You would want to weigh both the risk of taking action that a member or members would so seriously contest that they would go to court to try to block the outcome and your risk of losing the case.  Your violation of the rules is technical more than substantive.  You have certainly given your members substantially all of the time required by the notice provision.  A court might rule that you have “substantially complied” with the requirement, especially if the complainant can’t show any specific damage from the three-day error.

Most annual meetings don’t decide many issues of great importance, other than election of directors or perhaps officers.  Are any critical or controversial issues on your agenda?  If your meeting involves elections, is there a process by which members can vote without attending?  Are there any contested elections?

Anyone who attends the meeting without complaining about the notice is deemed to have waived the requirement.  If anyone objects to the lack of notice at the meeting, you can consider at that time whether you want to cancel it, especially if there is a vocal group of protesters.  Would those who don’t attend be so few in number that it would be practical to seek a waiver from those who aren’t there?  A waiver could be effective even if signed after the fact.

If you do get sued, you can always concede and hold a new meeting with the appropriate notice to accomplish what you did at the improper meeting.  You don’t intentionally want to operate your organization without following the rules.  But when there has been an innocent mistake, it may not make sense to cancel and start over if there isn’t likely to be any controversy or objection to what the members do.