The history of the United States is directly linked to the history of the Capital District. Not only is Albany the oldest active city charter still in operation today, innovations such as the telegraph, the electric motor, the steamboat, and the railroad were first established here. Many of us were taught in grade school of the importance of Fort Ticonderoga and the Battle of Saratoga in the French and Indian War and Revolution, but few people know of the area’s significance during the Civil War. In honor of Veterans Day, I wanted to explore the story of Elmer Ellsworth and his service and sacrifice at the beginning of the War Between the States.
Early Life in Upstate New York
Ellsworth was born in Malta, New York in 1837 to a family who immigrated to Saratoga County around the time of the Revolution. Elmer’s grandfather fought as a militiaman during the Revolution and his stories were passed on to young Elmer by his grandmother, who survived her husband for many years. These stories likely influenced young Elmer’s ideas of glory, honor, and service to something greater than himself.
The “Panic” of the 1830s was the first depression the country faced, and it was hard on many families; the Ellsworths were no different. Elmer’s father, an experienced tailor before the Panic, had to make a living by doing less-than-desirable tasks, such as catching pigeons and selling them for their meat. While he was in school (albeit intermittently), Elmer’s friends often teased him for his small stature and his father’s occupation—or lack thereof. Elmer learned to fight back early and never shied away from a confrontation.
Saratoga Springs was a popular resort town in the mid-nineteenth century (apparently not much has changed) and Elmer likely saw many young officers from West Point and Veterans of The Mexican War in full military regalia trekking north to take in the healing waters of The Spa. Elmer and his family moved to Mechanicville in the early 1840s, and his dreams of military greatness started to take hold. He organized some local boys into a “militia”, which he called the “Black-Plumed Riflemen of Stillwater”—a name he appropriated from a book he read about the Revolutionary War (Goodheart, 2012).
Elmer moved to New York City sometime at the end of the 1840s, then to Chicago in the early 1850s. In 1850, there were only about 30,000 people in Chicago but the city population exploded during that decade. Elmer worked as a clerk in law offices for minimal pay, barely eking out a living, but he always remained optimistic that one day his personal sense of grandeur would be recognized by other people. While living on a hardwood floor and eating only bread and water, he once wrote in his diary, “Am living like a King” (Goodheart, 2012).
In the mid-nineteenth century, militias were still common in the north and the south. Early Americans did not believe in having a standing army to provide defense since they could be used by the government to oppress the people; instead, they preferred coming together to conduct drills and trainings on a community level. In reality, the training and supply of these militias were seriously lacking. Many of these militiamen used these drill exercises as excuses to get together away from home and enjoy friendly company and some heavy drinking (Goodheart, 2012). But by the 1850s, and after America’s success in The Mexican War, a fresh sense of patriotism swept over these militias. Elmer was a member of one of these militias in the late 1850s and appears to have been more interested in the discipline of military service rather than the political upheavals of slavery that were hurtling the country toward civil war at that time. His Chicago unit elected him Major at the age of 19 and he reveled in commanding his “troops” in parades and drill ceremonies.
The U.S. Zouave Cadets
In the late 1850s, Major Ellsworth met a French fencing instructor named Charles DeVilliers who had served as an Algerian Zouave officer. The Zouaves were as well known for their fierce fighting tactics as for their flashy uniforms—loose jacket, baggy pants, and a fez. DeVilliers fought in Crimea, and newspapers reported the Zouaves’ successes in that conflict around the world. Ellsworth insisted on learning some of the Zouave officers’ fighting techniques, which included flashy bayonet drills that were very successful in the French army’s Algerian campaign. Major Ellsworth taught his soldiers these techniques, and they renamed their unit the U.S. Zouave Cadets. These cadets were beholden to strict rules so their physical fitness would not be compromised. They were prohibited from drinking in any saloon, gambling, or frequenting any “house of ill-fame” (Goodheart, 2012).
Chicagoans’ first look at the U.S. Zouave Cadets was at a Fourth of July parade. The parade-goers were stunned with what they saw; the military drill looked more like gymnastics with soldiers jumping, rolling, and yelling with loaded muskets (Goodheart, 2012), pausing occasionally to fire in unison. The choreographed movements amazed onlookers and the U.S. Zouave Cadets’ reputation began to spread across the country. They traveled across the Midwest, New York, and New England, showing off their techniques in front of massive crowds, sometimes as large as 25,000 people (Goodheart, 2012).
The result of U.S. Zouave Cadets’ rapid popularity was that Elmer Ellsworth became the country’s first “sex symbol.” His picture was mass-produced and distributed nation-wide, and young girls were known to swoon over his image while young boys attempted to imitate his movements and charisma (Goodheart, 2012).
Ellsworth and Lincoln
Elmer Ellsworth and Abraham Lincoln knew each other before either of them had risen to national prominence. They first met on the Illinois legal circuit, and Ellsworth saw Lincoln as a role model of sorts. They both came from humble beginnings and were each practicing in the law profession—Lincoln as a lawyer and Ellsworth as a law clerk. After the U.S. Zouave Cadets visited Springfield, Illinois to conduct a drill ceremony, Ellsworth decided to resign his command of the U.S. Zouave Cadets when they returned to Chicago to take up a clerkship with the future president. When Lincoln was elected president and left Springfield for Washington, Ellsworth joined his entourage and accompanied him on his journey east.
At the onset of the Civil War, Lincoln wanted to appoint Ellsworth to a post in the War Department overseeing the Union Army’s militias, but Ellsworth had other plans. He could not see himself at a desk job in Washington, but rather he wanted to contribute to the war effort firsthand. Much to the newly-elected president’s chagrin, his protégé went back to New York City after the fall of Fort Sumter and began assembling a new unit to fight the rebels.
The Fire Zouaves
Ellsworth was intent on raising his own elite unit of soldiers for the war effort similar to how he did the U.S. Zouave Cadets a few years earlier. This time, instead of law clerks and other professionals, he explicitly targeted New York City firefighters, since they were known to be ready for a fight at a moment’s notice. New York’s firefighters had a reputation for being ornery and ungovernable, often more apt to fight each other (and rival fire departments) than the fires themselves. Volunteers for Ellsworth’s new Zouave unit would be paid $13 a month and receive the highest-quality, flashy Zouave uniform that featured a bright red firemen’s shirt. Many of these firefighters saw Ellsworth and his U.S. Zouave Cadets parade through New York during their exhibition the previous summer, so they were more than ready to join his ranks in defense of the Union cause. Some called them “the B’hoys,” others called them “the Fire Zouaves,” but the new unit certainly stood out in their colorful, red Zouave shirts, and garish uniforms.
Once Colonel Ellsworth’s Zouave unit was raised, it received great fanfare when it left New York City enroute to Washington, DC. The Union Capital was in a precarious position, especially at the beginning of the war. The city was surrounded by slave-owning states, and only the Potomac River separated Washington from Alexandria—where the Confederate’s favorite son, Robert E. Lee, called Arlington home. The Fire Zouaves were officially sworn into the Union army, elected Ellsworth Colonel, and awaited their orders. At only 24 years old, Colonel Ellsworth spent most of his free time at the White House where he saw the Lincolns as surrogate parents. He played with Willie and Tad, the president’s two youngest sons, who wanted to emulate the Zouaves. They insisted on having their own uniforms made, complete with red sash and tassel.
Across the river from the White House, the president could see a Confederate flag flying high above the buildings. It was a constant reminder to Lincoln and everyone else on the Union side of the river that the army had not yet moved into Virginia to try and put an end to this secessionist nonsense— many thought the Union would easily win the war in a matter of weeks, not years. When Senator Benjamin Wade, a radical abolitionist from Ohio, met President Lincoln at the White House, he noted the flag’s constant presence. Lincoln told the senator not to expect to see it flying there for much longer. Colonel Ellsworth and his Fire Zouaves were deployed across the Potomac by steamship late at night on May 23, 1861 in what would be the first Union incursion into rebel-held territory.
Capture the Flag
Once the Fire Zouaves entered Alexandria, Colonel Ellsworth commanded one of his companies to destroy Alexandria’s railroad hub while he and a small team moved toward Alexandria’s telegraph offices. On their way to the telegraph offices, however, Colonel Ellsworth’s detail came upon the building that housed the very same Confederate flag that could be seen across the river in Washington. The flag flew above the Marshall House, a local tavern that doubled as an inn, and its proprietor, James W. Jackson, was known as an “ardent secessionist” (Goodheart, 2012). Jackson had a seamstress make a Confederate flag–the so-called “Stars and Bars”—to fly above his establishment and he added a star for every Confederate state that seceded.
Colonel Ellsworth sent off for reinforcements before entering the establishment himself with his two colleagues. At this late hour, early in the morning of May 24, most of the hotel’s guests were still sleeping. Colonel Ellsworth decided to go cut the flag down himself. He raced up the stairs to the roof where he used his bowie knife to free the building of the rebel flag. Colonel Ellsworth had the flag draped over his shoulders as he descended back into the Marshall House and proceeded back downstairs with his trophy.
Colonel Ellsworth and his two young soldiers rounded the corner of the second story only to run into James W. Jackson and his double-barrel shotgun. Jackson fired his weapon at point-blank range at Colonel Ellsworth who collapsed forward, still tangled in the folds of the rebel flag. Another shot rang out, this time in the direction of Jackson who began to fall before Corporal Brownell, one of Ellsworth’s soldiers, thrust his bayonet into Jackson’s body.
The first two casualties of the Civil War were claimed—one Union, one Confederate.
The Legacy of Elmer Ellsworth
Colonel Ellsworth’s death inspired many other northerners to join the cause for union. Losing a national figure like Ellsworth was enough to enrage many people to take the fight seriously. Confederate papers called Ellsworth’s death “the first dead Yankee of thousands” (Goodheart, 2012) and many northerners began to realize this conflict would be more tragic and bloody than they initially believed.
The Fire Zouaves laid Elmer Ellsworth’s body in state in the East Room of the White House before it was sent home to Mechanicville. His parents laid him to rest in an unmarked grave on a hillside so as not to attract too much attention from mourners; to this day, it is still uncertain exactly where he was laid to rest.
The Marshall House was demolished in the 1950s, and in its place stands the Alexandrian (formerly the Hotel Monaco). The Sons and Daughters of Confederate Soldiers erected a plaque on the site that stood until 2017 that read:
The Marshall House stood upon this site, and within the building on the early morning of May 24, 1861 James W. Jackson was killed by federal soldiers while defending his property and personal rights, as stated in the verdict of the coroners jury. He was the first martyr to the cause of southern independence. The justice of history does not permit his name to be forgotten. Not in the excitement of battle, but coolly, and for a great principle. He laid down his life, an example to all, in defence of his home and the sacred soil of his native state, Virginia.
We here in the Capital District are very proud of our colonial and Revolutionary history. But on this Veterans Day, I wanted to bring attention to a hero from Saratoga County whose name is often overlooked in the most tumultuous moment in our country’s history. Over 600,000 Americans died during the Civil War so it is easy to lose the sight of the trees in the forest. Let us remember all our Veterans this weekend and thank them for their service. And history should not permit any of their names to be forgotten for their sacrifices.
 Albany: One of America’s First Cities. (n.d.). Retrieved from http://www.albanyinstitute.org/albany-one-of-americas-first-cities.html
 History of Albany, NY. (n.d.). Retrieved from https://www.albany.org/things-to-do/albany-heritage-tourism/history/
 RevolutionaryWarAnimated.com Presents The Saratoga Campaign. (n.d.).
 Goodheart, A. (2012). 1861: The Civil War Awakening. New York: Vintage Books.
 Landers, J. (2016, September 27). In the Early 19th Century, Firefighters Fought Fires … and Each Other. Retrieved from https://www.smithsonianmag.com/smithsonian-institution/early-19-century-firefighters-fought-fires-each-other-180960391/