Martin van Buren. Jeff Hummel makes the case in a paper delivered at a conference I attended and spoke at myself 1998, while still a lad.
Here’s Murray Rothbard making the case for van Buren about a decade earlier, with the important caveat that he was unlibertarian on the Indian question. (via Robert Wenzel, EconomicPolicyJournal.com).