Alas, none of President Fillmore's quotes fall into that category. Still, he had a few pithy things to say. The following are all attributed to him.
On the hell of being a pre-pension ex-president (a situation not dealt with until the Former Presidents Act of 1958):
"It is a national disgrace that our Presidents, after having occupied the highest position in the country, should be cast adrift, and, perhaps, be compelled to keep a corner grocery for subsistence."
On why he wasn't an abolitionist:
"God knows that I detest slavery, but it is an existing evil, for which we are not responsible, and we must endure it, till we can get rid of it without destroying the last hope of free government in the world."
A prescient thought indeed for the mid-19th century:
"England at present wields the destinies of the commercial world, and her power is concentrated in London; but if this country can maintain its union, there are those now within the hearing of my voice who will live to see New York what London is now."
Something to gladden the heart of Ron Paul:
"The government of the United States is a limited government. It is confined to the exercise of powers expressly granted, and such others as may be necessary for carrying those powers into effect; and it is at all times an especial duty to guard against any infringement on the just rights of the states."
On the 19th-century locusts known as office-seekers:
"Nothing brings out the lower traits of human nature like office-seeking. Men of good character and impulses are betrayed by it into all sorts of meanness."
In an antebellum moment of despair:
"May God save the country, for it is evident that the people will not."
Sure, I headed up the Know-Nothing Ticket in '56, but some of my best friends are foreigners:
"I have no hostility to foreigners. . . . Having witnessed their deplorable condition in the old country, God forbid I should add to their sufferings by refusing them an asylum in this."