From the Journal of the Senate 2nd Session, 37th Congress:
Wednesday, December 4, 1861
Mr. Chandler submitted the following resolution for consideration:
"Resolved, That John C. Breckinridge be, and he hereby is, expelled from the Senate."
The Senate proceeded, by unanimous consent, to consider the resolution; and the same having been amended, on the motion of Mr. Trumbull, to read as follows:
"Whereas John C. Breckinridge, a member of this body from the State of Kentucky, has joined the enemies of his country, and is now in arms against the Government he had sworn to support: Therefore,
"Resolved, That John C. Breckinridge, the traitor, be, and he hereby is, expelled from the Senate."
Breckinridge wasn't around for his expulsion. Fearing arrest, he had already skedaddled behind Confederate lines. "I exchange with proud satisfaction a term of six years in the United States Senate for the musket of a soldier,” he wrote.
He became an important general in the service of the CSA, seeing action at Shiloh, Stones River, New Market and Cold Harbor, among other places. In the last months of the war, he was the final Confederate Secretary of War. After the war he spent a few years in exile, but returned the Kentucky after President Johnson's unconditional Christmas Day amnesty in 1868, resuming his legal work and becoming a railroad executive. He died in 1875 at a relatively young 54.
More on Breckinridge's decision to go south is at Disunion, the New York Times blog about the Civil War, 150 years after it happened.