“The phone tapping, human collection, and aerial snooping today’s U.S. spy community engages in can be traced back to the Civil War.”

By Douglas Waller

ABRAHAM LINCOLN WAS one of the least experienced men to assume the presidency, but he was hardly a neophyte when it came to the dark arts of intrigue and subterfuge.

As an ambitious politician in Illinois, he often wrote newspaper columns under aliases attacking his political opponents and at one point he secretly bought a German-language newspaper to print puff pieces about himself him for that important voting bloc in the state. Later, during his race for the presidency, he was a careful reader and evaluator of political intelligence.

Once in the White House, Lincoln ordered the army to deliver him daily intelligence reports on the enemy and had freelance agents all over the country send him information on the Rebels and their sympathizers. He had no qualms about launching risky covert operations against the South and he found subversion and propaganda useful tools to undermine the border states that had joined the Confederacy and to keep the ones that remained with the Union under federal control.

Ultimately, Lincoln and the Union Army had at their disposal a robust network of secret agents infiltrating the South, aeronauts observing enemy movements from hydrogen gas-filled balloons, photographers taking shots of future battlefields and signal corps officers intercepting enemy information transmitted by flag or over telegraph lines. The phone tapping, human collection, and aerial snooping today’s U.S. spy community engages in can be traced back to the Civil War.

Here are four agents who spied for Lincoln’s Union Army in the all-important Eastern theatre of the Civil War:

Allan Pinkerton

Friends said Allan Pinkerton was gifted with unusual powers of observation. Born in Scotland, as a young man he trained to be a barrel maker, but ended up spending more time working as a labor agitator. In 1842, Pinkerton emigrated to America with his young wife, eventually settling in Chicago, where he founded a private detective agency in the mid-1850’s that became highly successful.

As a detective he honed a sixth sense to anticipate criminal activity before it happened. He was stubbornly persistent, refusing to be worn down by adversity. After the Civil War began in spring 1861, Pinkerton, who by then had become famous nationwide as a private eye, joined Union General George McClellan’s force to serve as his spymaster.

Pinkerton used women to infiltrate Rebel social circles, recruited runaway slaves to collect information, infiltrated spies into Richmond, and succeeded in breaking up a Confederate espionage ring in Washington. But Pinkerton ended up being a failure as a battlefield intelligence officer. He fed McClellan wildly inaccurate reports that intentionally inflated the number of Rebel troops the Union general faced. Pinkerton also spied on Lincoln for political intelligence he thought might be useful for McClellan, who mounted his own run for the presidency in 1864.

After the war, Pinkerton’s multi-million-dollar National Detective Agency hired out guards and spies for strikebreaking robber barons, caught pilfering employees and hunted bandits like the Reno brothers and the Jesse James-Cole Younger gang. Pinkerton, who wrote 18 books on his crime fighting and Civil War exploits, died in 1884 at the age of 64, but his agency continues to operate to this day.

Lafayette Baker

Lafayette Baker, a handsome man, with brown hair, a full red beard, and piercing gray eyes, was a fine horseman and a crack shot. Fed up with his father, an insufferably stern Puritan, Baker renounced God and ran away from his Michigan home in his late teens, largely uneducated save for a little schooling that taught him to read and write. For the next ten years he drifted from job to job through a dozen states, often having to flee a city after getting into trouble with locals. He finally ended up in San Francisco by the mid-1850’s, joining a vigilante group that rounded up suspected criminals in the lawless city. When the Civil War started, Baker, who was back on the East Coast, rode into Washington, hoping to land a good-paying job with the Union Army. He managed to talk General Winfield Scott, the Army’s aging commander, into hiring him as a secret service agent.

Baker eventually set up his headquarters in an old two-story brick building on Pennsylvania Avenue near the U.S. Capitol. He commanded some 30 detectives who put hundreds of suspected spies and thieves in jail, while pocketing cash from the graft they uncovered. Baker liked to brag that there was no hostile agent in Washington that he or his men didn’t know about. But he failed to uncover the threat to Lincoln in the city from John Wilkes Booth and his gang.

Baker, who had a falling out with War Secretary Edwin Stanton and later President Andrew Johnson, accused the two men of being involved in the plot to kill Lincoln. Congressional investigators found no evidence to back up Baker’s claims and dismissed him as a liar. His savings wiped out in a failed Michigan hotel venture, Baker wrote a self-serving memoir titled History of the United States Secret Service. The title was misleading because his agency was not the forerunner of today’s U.S. Secret Service. Disgraced and dejected, in 1868 he died of typhoid fever in Philadelphia at age 42.

George Sharpe

His superiors considered George Sharpe a natural military leader, with a magnetic personality that made men want to follow him. In the breast pocket of Sharpe’s uniform coat he kept a small, well-thumbed book of verses by his favorite poets, which he routinely read to his men. Sharpe was born in Kingston, New York on the Hudson River. The son of a wealthy merchant, he attended elite academies as a youngster, graduated from Rutgers University with honors, and earned a law degree from Yale University. Before setting up his practice, Sharpe spent four years in Europe, studying French in Paris and working as a secretary in the U.S. legations in Vienna and Rome. When the war broke out, he first commanded a company of federal militiamen and later led an infantry regiment as a colonel. The duty prepared Sharpe for the most important job he would have as the Union Army’s preeminent spymaster in early 1863.

Sharpe pioneered what spy agencies today call “all source intelligence.” His bureau raked in information not only from Sharpe’s spies, but also from the reports of prisoner interrogations, from aeronauts flying in balloons over the battlefield, from signal officers intercepting Rebel telegraph messages, and from cavalry scouts on reconnaissance patrols. His officers then sorted and analyzed this flood of information to produce for Union commanders the most accurate and comprehensive picture of the enemy that they had ever had.

Unlike Pinkerton and Baker, Sharpe never wrote a memoir. In all the speeches he gave on the war years before civic groups and reunions of his infantry regiment, he almost never talked about the spy organization he led. Sharpe resumed his law practice in Kingston, won a seat in the New York Assembly becoming the legislature’s speaker, and was appointed by President Grant to several high-level federal positions in New York City. He died in 1900 at the age of 81 and was given a war hero’s funeral in Kingston.

Elizabeth Van Lew

Elizabeth Van Lew’s father was a wealthy Richmond, Virginia hardware merchant. Her mother was a highly educated socialite, who stocked the library of their mansion with almost 600 books. Van Lew, who developed an early empathy for slaves, was sent to relatives in Philadelphia to be educated. A governess there lectured her on abolitionism and she returned to Richmond with an even fiercer hatred of human bondage. She was a short woman, who had been quite beautiful in her youth. But when the Civil War started, Van Lew was in her 40s and unmarried—considered by Richmond society to be an old maid. When Union prisoners began pouring into the city, Van Lew cajoled Confederate authorities into letting her bring meals and books to the POWs. It soon made her a pariah in her city.

But the Virginia heiress would not be cowed. She moved quickly from aiding Union war prisoners to collecting intelligence on the Rebels, at first sending unsolicited letters to Washington with her observations of conditions in Richmond. Van Lew’s undercover work soon came to the attention of General Benjamin Butler, the commander of the Union’s Department of Virginian, who had her send him intelligence written in invisible ink.

After Sharpe took over spy operations for the Army of the Potomac, he became Van Lew’s handler. The Virginia heiress was Sharpe’s best operative in Richmond, organizing a sophisticated spy ring that Confederate security agents were never able to crack. Van Lew’s network churned out intelligence for Sharpe on Richmond’s defenses, the condition of Robert E. Lee’s army, Rebel troop movements through the Confederate capital, economic conditions in the city, and the morale of its residents. With each delivery of coded messages, Van Lew also included for General Ulysses S. Grant the latest editions of Richmond’s newspapers plus a rose picked from her garden.

Sharpe boasted that whatever Grant wanted in the way of information from the Confederate capital, Van Lew’s ring could provide it. “She represented,” Sharpe wrote, “all that was left of the power of the U.S. government in the city of Richmond.”

Neighbours shunned Van Lew after the war. As president, Grant made Van Lew Richmond’s postmaster, a politically powerful job that paid her $4,000 a year. But succeeding presidents refused to rehire her for the position and by the late 1800s, Van Lew had depleted her fortune by her helping relatives and former slaves. She died nearly broke in 1900 at age 81.

ABOUT THE AUTHOR: A former correspondent for Newsweek and TIME, Douglas Waller has written seven books on military and intelligence history. His latest book, LINCOLN’S SPIES, is being released this month by Simon & Schuster.