Well, as indicated above, its the opinion of the Father of the Constitution, so while his opinion is not law, no, he's not "just one American among many."Madison's opinion is not law. He may have held the preference that the Constitution had to be "adopted in toto and forever", but he was just one American among many, with no more power to enforce his ideas than anyone else had to enforce theirs.
It's just an opinion, nothing more. Others held different opinions, just as valid as Madison's.
And if anything, we have in the ratification debates what would have been for federalists an irresistible opportunity to calm ALL antifederalist concerns that states could withdraw unilaterally at will, if such had been the case. Not one federalist said any such thing. Instead, we get "in toto, and for ever," and:
“The Constitution must be the supreme law of the land; otherwise, it would be in the power of any one state to counteract the other states, and withdraw itself from the Union.” — Governor Samuel Johnston, NC ratification convention, July 29, 1788