At the outbreak of the Civil War, he enlisted in the Texas militia and was named captain of Company B, Eighth Texas Cavalry. When the commanders of the unit, Benjamin Terry and Thomas Lubbock, died within weeks of each other, he was named commander of the group - now named Terry's Texas Rangers after their organizer. He led his men through a series of exceptional battles: the Battle of Shiloh, where he was wounded; the 1862 Kentucky invasion, which led to his promotion to brigadier general; Murfreesboro, where he was wounded a second time; the Battle of Chickamauga, where he was promoted to major general in 1863. He was transferred to the Trans-Mississippi department in Louisiana in early 1864, leading the cavalry at the tail end of the Red River campaign. On April 6, 1865, while he was visiting General John Magruder's headquarters in Houston, General Wharton was shot and killed by another officer, George Wythe Baylor, over a debate in military tactics. He was first buried in Hempstead near his home, but was later re-interred with full honors in the Texas State Cemetery in recognition of his service.