Not a great deal of information is easily available about Ambrose, third child of the elder James Madison and Nell Rose Conway Madison, whose first child was James, born in 1751. All together, the Madisons had 12 children, not all of whom – as to be expected for the time – survived infancy.
Presumably Ambrose, who was named after his and James’ grandfather, grew up in similar circumstances as the younger James and the other surviving children of the elder James and Nell, as part of the landed gentry of Virginia in the late 18th century. He didn’t live long enough to see his brother become chief magistrate of the new republic in 1809, dying in 1793. At that time James Madison was a member of the House of Representatives.
Time, then, to spare a thought for presidential siblings, who are mostly historical nonentities. With certain exceptions, of course: Robert Kennedy was in his brother’s cabinet and might have become president himself; Donald Nixon invented the Nixonburger; and Billy Carter gave the world Billy Beer.