Although nullification itself was not discussed much at the time of the Constitution’s drafting and ratification, the consensus at that time was that authoritative decisions about the constitutionality of federal laws would be the province of the federal courts, not the states.
This was the view of an overwhelming weight of authorities and experts. Both the Constitutional Convention, the state ratifying conventions, and over half a dozen Federalist Papers show clearly that the founders of our Constitution attributed this authority only to the federal courts. And that implied that nullification was illegal.
But what should the states do, if Congress steps way out of its constitutional bounds and the Judicial branch colludes (or could be expected to do so) in the usurpation? That issue raised its head a mere eleven years after the Constitution was written — with the passing of the unconstitutional and quite tyrannical “Alien and Sedition Acts”. Both James Madison and Thomas Jefferson were extremely disturbed by this legislation.
In his Virginia Resolution (passed Dec. 24, 1798), a response to those acts, Madison floated the idea of a state’s “interposing” against an unconstitutional federal law. This Resolution did not indicate, though, what concrete acts “interposition” might involve. Later, in the Report of 1800, Madison gave some specific examples: communicating with other states about the unconstitutional federal law, attempting to enlist the support of other states, petitioning Congress to repeal the law, introducing Constitutional amendments in Congress, or calling a constitutional convention. None of these is very surprising — much less, of questionable legality.
In Paper 16, Hamilton distinguished carefully “between a mere NON-COMPLIANCE and a DIRECT and ACTIVE RESISTANCE. If the interposition of the State legislatures be necessary to give effect to a measure of the Union, they have only NOT TO ACT, or TO ACT EVASIVELY, and the measure is defeated.” Hamilton continued:
But if the execution of the laws of the national government should not require the intervention of the State legislatures, if they were to pass into immediate operation upon the citizens themselves, the particular [i.e., state] governments could not interrupt their progress without an open and violent exertion of an unconstitutional power…. They would be obliged to act, and in such a manner as would leave no doubt that they had encroached on the national rights. An experiment of this nature would always be hazardous in the face of a constitution in any degree competent to its own defense, and of a people enlightened enough to distinguish between a legal exercise and an illegal usurpation of authority…. Attempts of this kind would not often be made with levity or rashness, because they could seldom be made without danger to the authors…. (Interpolations and emphasis ours.)
Unfortunately, in the face of the Alien & Sedition Acts, for a very brief time in 1798-99 Thomas Jefferson succumbed to the temptation to lay that very course. The “Kentucky Resolutions” of 1798 were likewise a state’s response to those acts. Jefferson, who was Vice-President, was their original author, though this was not widely known at the time. (Jefferson and Madison were close friends and fellow members of the Democratic-Republican Party, and carried on extensive correspondence during this period. Each wrote his Resolutions anonymously.) It was in the Kentucky Resolutions that Jefferson coined the term “nullification”.
The Kentucky Legislature itself was more reserved in their statements than was Jefferson: before they passed his Resolutions they removed the term “nullification” (although they restored it in their 1799 version). Also, their 1798 version ended by expressing the hope “… that the Co-states recurring to their natural right in cases not made federal [by the Constitution], will concur in declaring these [Alien and Sedition] acts void and of no force, and will each unite with this Commonwealth in requesting their repeal at the next session of Congress.”
Jefferson’s original drafts were more straightforward, about what “nullification” truly involved: in those drafts the same passage reads:
… and that the co-states, recurring to their natural right in cases not made federal, will concur in declaring these acts void and of no force, & will each take measures of it’s own for providing that neither these acts, nor any others of the general government, not plainly & intentionally authorised by the constitution, shall be exercised within their respective territories. (Emphasis added.)
Here Jefferson went boldly where Madison feared to tread: what Jefferson meant by the term, and what he hoped for from the other states, involved not only the state’s pronouncement of the “null and void” status of an unconstitutional federal law, but also the state’s taking whatever measures are necessary to make sure the law is not enforced within its boundaries.
This is the whole point of nullification. But practically, in such cases as Jefferson had in mind, those “measures” would entail the use of force – or at least, the credible threat of it.
Jefferson was fully aware that it might involve actual violence. On Nov. 17, 1798, after he had finished his draft of the Kentucky resolves, he wrote in a letter to Madison, “I inclose you a copy of the draught of the Kentuckey resolves. I think we should distinctly affirm all the important principles they contain, so as to hold to that ground in future, and leave the matter in such a train as that we may not be committed absolutely to push the matter to extremities, & yet may be free to push as far as events will render prudent.” (Emphasis added.)
By early 1799, however, Jefferson had decided that any actual use of violence would be counterproductive — for some of the same reasons Hamilton had mentioned years earlier. In a letter to Madison on Jan. 30, Jefferson wrote that “Firmness on our part, but a passive firmness, is the true course. Anything rash or threatening might check the favorable dispositions of [several States that had petitioned against the Alien & Sedition Acts], & rally them again around the measures which are ruining us.” (Emphasis added.)
Instead, in late August 1799 Jefferson suggested a different radical remedy — secession. In letters to Madison and to Wilson C. Nicholas, he outlined his new plan for strengthening public support for the principles expressed in the resolutions.
Jefferson suggested that a new set of resolves be passed by the legislatures of Kentucky and Virginia, and sent to the various states, which, after encouraging their support for the resolutions, would add that:
[Kentucky and Virginia were] fully confident that the good sense of the American people and their attachment to those very rights which we are now vindicating will, before it shall be too late, rally with us round the true principles of our federal compact; but determined, were we to be disappointed in this, to sever ourselves from that union we so much value, rather than give up the rights of self government which we have reserved, & in which alone we see liberty, safety & happiness. (Emphasis ours.)
This idea horrified Madison, it seems. He paid a visit to Jefferson the very next week, and convinced him to back down from his plan.
Most serious Jefferson historians, as well as most other American leaders at the time, have been dismayed by his actions during this period. As Wikipedia points out,
In writing the Kentucky Resolutions, Jefferson warned that, “unless arrested at the threshold,” the Alien and Sedition Acts would “necessarily drive these states into revolution and blood.” Historian Ron Chernow says of this “he wasn’t calling for peaceful protests or civil disobedience: he was calling for outright rebellion, if needed, against the federal government of which he was vice president.” Jefferson “thus set forth a radical doctrine of states’ rights that effectively undermined the constitution.”  … George Washington was so appalled by [the resolutions] that he told Patrick Henry that if “systematically and pertinaciously pursued,” they would “dissolve the union or produce coercion.”  (Emphases ours; interpolation Chernow’s.)