Ellie Swain:

The Forbidden Love That Rocked The South And Closed A University
By Ted Sampley
Olde Kinston Gazette
February 1999 Issue

Ellie Swain was only 17 when North Carolina seceded from the Union in 1861. She had always been graced with the refined life of an aristocratic southern belle. She lived with her family in Chapel Hill, a subdued little village nestled in rolling wooded hills 30 miles west of North Carolina's capital city of Raleigh. The village dated before the American Revolution. It drew its name from the of ruins of a chapel built there in the colonial days by the Church of England.

Ellie Swain

Spoiled by an overprotective mother and pampered by family slaves, Miss Ellie spent her girlhood in a culture punctuated with all sorts of "gay doings, jokes, serenades, picnics, wild-flower hunts and autumn excursions after hickory nuts and wild grapes."

But the black clouds of war changed it all.

When the first cannon thundered in 1861, how could anyone have known, or even guessed, that the charming and lovely Ellie would become involved in a scandalous romance--the consequences of which would tear apart the polite society of Chapel Hill and cause much bitterness and a great upheaval throughout all of North Carolina.

Ellie grew up amongst the elite world of academia on the campus of the University of North Carolina. The university had been established in 1795, and her father David Lowry Swain moved the family to Chapel Hill in January 1836 after receiving an appointment as president of the university.

Before becoming an educator, Ellie's father had been a lawyer and served two terms as governor of North Carolina. He remained head of the university for more than 30 years. Ellie's grandmother on her father's side was Carolyn Swain, the daughter of Jesse Lane, a member of a prominent North Carolina family.

Ellie's mother Eleanor, for whom she was named, was the daughter of former Secretary of State William White. His wife Anna was the daughter of Richard Caswell, a Revolutionary War hero and North Carolina's first constitutional governor.

In February 1861 just before the outbreak of hostilities, Ellie watched her father leave the university to lead a delegation sent by the North Carolina state legislature to Montgomery, Alabama to meet with representatives of southern states that had seceded from the Union. The 1860 election in North Carolina had shown that an "overwhelming majority of North Carolinians were unionist -- opposed to secession," and he had hopes of finding some compromise which could lead to reconciliation with the federal government.

By the time the Swain delegation arrived in Montgomery, the Confederacy had been formed. Instead of the Confederates listening to talk about reconciliation, they lobbied for the North Carolinians to join the Southern cause.

The delegation returned to North Carolina to report its failure.

On April 12, Confederate troops fired on the federal garrison at Fort Sumter in the harbor of Charleston, South Carolina, capturing it two days later.

President Abraham Lincoln issued a national call on April 15 to the states in the union for 75,000 troops to suppress the "southern insurrection," two regiments of which were required from North Carolina.

North Carolina refused Lincoln's demand for troops and countered by seizing all United States forts and property within the state.

On May 20, North Carolina assembled a convention in Raleigh with 120 delegates present and seceded from the union.

It did not take long before most residents of North Carolina found themselves at the mercy of a federal blockade and struggling from shortages of food and clothing while having to grieve the loss of loved ones who died after going off to fight the Yankees.

The die of secession having been cast, Ellie's father and other old unionists solemnly accepted "the tragedy" they had tried in vain to avert. Swain became "a loyal though reluctant supporter" of North Carolina's effort in the war for "Southern Independence." He began devoting most of his time to keeping the university operating, using as much of his influence as possible in attempts to gain exemption from conscription for university students and faculty.

The effort was to no avail, however. Most of the students and younger faculty members volunteered to fight for the Confederate cause.

To Ellie's dismay, this left only a few students who were either too young to enlist or exempt because of ill health or war injuries. The only professors left on campus were her father and others who were too old to serve in the military. For a young southern belle just entering courting age, Chapel Hill had quickly become lacking in appropriate suitors.

Ellie's family managed to remain shielded from the harshness of the bloody war which was raging all around them. People living outside the campus were not so lucky. Tens of thousands of Southern men and boys were being killed and wounded on battlefields far from their homes. Chapel Hill was often plunged into mourning as reports arrived listing the local boys who had died or were missing.

By the end of the war between the states, nearly every household in the South had either lost someone or knew someone who died in the war.

Despite the hardships, Ellie's father worked long hours and managed to keep the university open. Every year of the war, he continued to hold commencement exercises.

By early April 1865, it had become obvious to the residents of Chapel Hill that their little village was about to be captured by the Yankee forces of General William T. Sherman. His Federals had already burned and plundered a path from Georgia through South Carolina and into North Carolina.

On April 9, once again Ellie watched as her father left Chapel Hill on a diplomatic mission - this time he rushed to Raleigh to meet with North Carolina's Confederate Governor Zebulon Baird Vance.

Upon Swain's arrival in the capital city, Governor Vance appointed him to a peace commission and on April 12 sent him under a flag of truce through the Confederate lines to Gen. Sherman's headquarters. Gov. Vance wanted a personal interview with Sherman to work out terms for the surrender of Raleigh.

The commission was successful in negotiations with Sherman. He agreed to meet with Governor Vance and to spare Raleigh. Swain took advantage of the opportunity and managed to get Sherman's pledge to spare Chapel Hill and its university.

On the return home, the commissioners were stopped by Sherman's cavalry commander, Gen. Hugh Judson Kilpatrick, five miles from Raleigh. He informed the commissioners they could proceed under the flag of truce, but issued a firm warning that "we will give you hell" if any resistance is met in Raleigh.

The commissioners were well aware that Gen. Kilpatrick's threat was not to be taken lightly. His cavalry was responsible for most of the devastation that had taken place in Georgia and South Carolina.

Shortly before the cocky Yankee general had begun his march into South Carolina, he bragged to his officers: "In after years when travelers passing through South Carolina shall see chimney stacks without houses, and the country desolate, and shall ask 'Who did this?' some Yankee will answer ' Kilpatrick's cavalry.'"

When the peace commission did not return on time, Governor Vance, fearing the worst, joined Confederate Gen. Joseph E. Johnson's troops as they retreated through Raleigh.

On April 13, Swain arrived in Raleigh to find the city nearly deserted. No one could be found on the streets and all the shops were closed. The governor and all the state officials had gone.

Within hours, General Kilpatrick rode into Raleigh at the head of his division with banners and guidons unfurled and bands playing.

Suddenly there was a resounding "--- damn 'em," followed by five quick shots which scattered Kilpatrick's troops. A rash young Texan from Confederate General Joseph Wheeler's command, who had lagged behind, had emptied his revolver at the approaching federals. After firing the shots, the Texan, named Walsh, wheeled and spurred his horse, galloping up Morgan Street closely pursued by a dozen federal horsemen. Turning a corner, Walsh's horse fell, and he was overtaken and brought back to Gen. Kilpatrick, who ordered his immediate execution.

Swain, who was watching from the Capitol, froze.

Despite pleas to Kilpatrick from the young Confederate soldier to withhold the execution for five minutes so he could write his wife, Walsh was executed on the spot.

Gen. Kilpatrick then assured Swain that he did not hold the young Texan's act of resistance against the city of Raleigh, and that there would be no reprisal. Swain was eventually allowed to return to Chapel Hill.

Before the dust had cleared from the Confederates retreating through Chapel Hill early Easter Sunday, residents began hiding their silver, linens and other personal valuables. They had heard the reports of Sherman's vicious war against the civilian population and were expecting the worst. The village was now completely at the mercy of the advancing enemy.

Ellie's father was confident that he had done everything possible to save the university and hopefully Chapel Hill. Plundering was expected of the federal troops, so he, too, took precautions to hide those things most dear to him - that is, everything except his precious and exceptionally charming 21-year old daughter Ellie.

Late Easter afternoon, a small attachment of "Yankee blue jackets" rode into Chapel Hill, soon followed by one of Gen. Kilpatrick's cavalry brigades commanded by Brigadier General Smith D. Atkins.

To the relief of the residents of Chapel Hill, Yankey cavalryman Capt. J.M. Schermerhorn informed them that he had been sent there to protect the university.

Cornelia Phillips, a renowned citizen of Chapel Hill and neighbor of the Swains, noted in her diary the events that took place upon the capture of the village.

According to excerpts from the diary, the next day, Gen. Atkins, handsomely attired in all his military trappings, made a formal call on Ellie's father to discuss quartering the 4,000 cavalrymen under his command.

During the meeting with Atkins, which quickly became cordial, Swain mentioned that he possessed the order book that had belonged to Lord Cornwallis in his march through North Carolina during the Revolutionary War. When the invader general showed interest in the relic, Swain offered to show it to him and asked Ellie, who was nearby already bursting her seams with curiosity, to fetch it.

"The young lady did so, perhaps not unwilling to have a look at the Yankee general," Mrs. Spencer wrote. "She threw up her head and marched in with great display of hauteur. An introduction was unavoidable, which was more than the governor had intended. They 'changed' eyes at first sight, and a wooing followed on that first meeting which greatly incensed all who looked on, including the Federal army and gave Governor Swain and his wife as much uneasiness as anything short of a death in the family could have done."

The Yankee general was captured. The blue eyed southern belle had shot a mystical arrow straight into his heart where no Confederate mini ball had been able to pierce. He immediately began endeavoring to win both her and her family's affection.

Every evening the general ordered his regimental band to play in front of the Swain home.

When Gen. Sherman gave Ellie's father a fine carriage and horse in a gesture of good will, Gen. Atkins presented Ellie with a superior riding horse. Instead of considering the concerts and gifts a compliment to Ellie's father, the people of Chapel Hill knew that it was all to please Ellie.

Tongues began to wag. The gifts were resented. The horses, the villagers claimed, had been stolen by the Yankees from other families in the South. How could the daughter of Chapel Hill's most prominent citizen fraternize with a hated Yankee whose sword dripped with Confederate blood and whose troops had looted and burned their way through the South?

The horse was stolen from Swain three times, and Swain stubbornly recovered it each time.

Even Gen. Atkins' own troops expressed displeasure at their commander's blatant display of affection. This was something a Northern general absolutely would not do.

The lovers paid no attention, and the courting continued.

On May 18, eighteen days after his arrival in Chapel Hill, Gen. Atkins received orders transferring him to a post in western North Carolina.

After the general had left, Miss Ellie handed her parents a note in which she formally told them that she had promised to marry the general and reminded them that she was 21 and old enough to make such decisions for herself.

Ellie's father was distraught. His whole mind and thoughts had been concentrated on protecting the university. This was something he had not expected.

At home, Ellie cried, quarreled and pleaded. Her father knew that such a wedding would send shock waves throughout North Carolina, but he was not accustomed to denying his children anything they wanted. Consenting to such a wedding was a bitter pill to swallow. He began to reason that such a marriage would be the first of its kind and could be the beginning of a healing process between the North and the South.

"Governor Swain believed this marriage," Mrs. Spencer wrote, "was but the first of many others like it to take place all over the South: that our peace was to flow like a river, and that North and South were coming together at once, to be more firmly united than ever. . . He did not once dream of the party issues that were to spring up and divide the country even more effectively than the war, nor the bitterness that was to be engendered and revived."

He traveled North to Freeport, Illinois to investigate the character of his soon to be son-in-law. There he found that Gen. Atkins was respected and from a good family. Before leaving to fight against the Confederacy, Atkins had been editor of a prosperous newspaper.

Swain returned home and consented to the marriage.

When announcement of the marriage was made, North Carolinians responded with indignation. Some even spat upon the wedding invitations.

Mrs. Spencer noted in her diary:

"We went to E.H. Swain's wedding Wednesday night, 23rd [August 1865]. Married in the face - in the very teeth - of all this bitterness and woeful humiliation, to the Yankee general. . . Very, very few people went to the wedding, tho' very general invitations were issued and a grand supper prepared. . . Gen. Atkins is a handsome man, rather grave in expression, sedate and courteous in manner. Elly looked well - beautifully dressed."

Local citizens and students at the university who were outraged at the blasphemy that was about to take place began ringing a makeshift church bell and the university bell from the time the bridegroom put his foot in the stirrup to go to the wedding until well after the ceremony. The university bell rang all night and students hanged Swain and Gen. Atkins in effigy. Overnight the university had lost many of its old friends and gained many new and powerful enemies.

From that day forth, everyone connected with the wedding, even the minister, was treated with repulsion.

Seven days later, Ellie and her general packed up and left for a happy and quiet family life in Illinois.

The university was denounced by the Northerners as a stronghold of "unreconstructed rebels" and Southerners claimed that the marriage proved the university was the center of "Unionism and disloyalty."

Mrs. Spencer, as if grasping for reasons as to why the marriage took place, wrote: "The only way one can find an apology for it all is in believing honestly in the love which appears to have brought it about. Let us speak respectfully of a genuine love affair."

Another entry in Mrs. Spencer's diary stated: " This marriage was of ill omen to Governor Swain,. The blight that fell immediately on the university was directly attributable to the fact that he not only permitted his daughter to marry an invader, but that he gave her a fine wedding. It was told from mouth to mouth and believed all over North Carolina that Ellie Swain went to Illinois loaded with finery and jewels stolen from states farther south, and given to her by her husband."

North Carolinians and a host of Northerners began clamoring for Swain's resignation from the university. Swain put off his resignation, hoping the situation would get better. It did not.

Repercussions of the controversy surrounding the marriage were deadly to the university. Enrollment dropped drastically. There was no money allocated by the government, and buildings were falling into disrepair.

Three years later in 1868, W.W. Holden, who had been appointed governor of North Carolina by the federal government, sent a militia of black soldiers to take possession of the campus, which he believed was a hotbed of insurrection against the federal government.

On July 23, 1868, Gov. Holden ordered Swain fired as President of the University of North Carolina.

The following month, on August 11, Ellie's father had his famous Sherman horse and buggy hitched and was taking a ride with a fellow professor when the horse spooked and bolted, throwing him and his companion from the buggy.

For more than two weeks, Swain lay terribly bruised and shaken. On the morning of August 27, he suddenly fainted and could not be revived.

With the former president's death, so went all his efforts to keep the university operating. It was finally closed in 1871 and not reopened until 1874.

As for Ellie and her family, when her children later visited their North Carolina kinfolk, the bitterness had disappeared, and they were welcomed home.

One of Ellie's several children later attended St. Mary's Junior College in Raleigh where she met and married a North Carolinian.

Sources: Sherman's March Through The Carolinas by John G. Barrett, The First State University by William S. Powell, Old Days In Chapel Hill by Hope Summerell Chamberlain, The Women Who Rang The Bell by Cornelia Phillips Spencer, Directory Of North Carolina Biography by William S. Powell and State Magazine.

Home