Fort Jefferson was essentially a very small and densely populated city. Almost 2,000 people lived there during the Civil War years just before Dr. Mudd arrived. The peak military population was 1,729, but there were also many civilians, a fluctuating number of military and civilian prisoners, and a few slaves, living there. They were all engaged in the construction and maintenance of the fort. Occupations included machinists, carpenters, plasterers, bakers, butchers, painters, blacksmiths, masons, and general laborers. There were also lighthouse keepers and their families, cooks, a civilian doctor and his family. A number of officers brought their families, including children, and a limited number of enlisted personnel brought wives who served as laundresses (typically four per company). The 22 slaves working at Fort Jefferson in 1863 were freed shortly after Lincoln’s Emancipation Proclamation.

With the war over, the population of Fort Jefferson began to decline and had dropped to 1,013 by the time Dr. Mudd arrived in July 1865. Of these, 486 were soldiers or civilians and 527 were prisoners. When Dr. Mudd was released in March 1869, the population had dropped to 282, of whom only 35 were prisoners. 

The military prisoners at Fort Jefferson were a rough crowd. Their offenses included murder, manslaughter, robbery, grand larceny, and desertion. In June 1867, Post Commander George P. Andrews issued Special Order No. 78 which said:

The attention of the officers of this Post is called to the fact that atrocious crimes have been committed by prisoners at this Post who seem to think they cannot be reached by law.

In future every sentinel must use his bayonet and cartridge, and no sentinel who faithfully tries to do his duty shall ever see the inside of the guard house; if a prisoner refuses to obey orders the sentinel must shoot him, and then use his bayonet, at the same time calling for the guard. The responsibility for obedience to this order will be borne by the commanding officer.

Daily life at Fort Jefferson was dictated by the rhythm of a busy army post. In the morning, bugle calls announced the time to rise and shine, to assemble for morning roll call, and for breakfast. The morning gun was fired as the flag was raised over the fort. Lunch, supper, and various events in between were announced by bugle calls. In the evening, bugle calls announced retreat, evening roll call, and bedtime taps. The evening gun was fired as the flag was lowered for the night. During the night, fort sentries loudly announced the regular changing of the guards. Unfortunately for Dr. Mudd and his companions, they had front row seats to the sights and sounds of all this activity. For most of their time at Fort Jefferson, they lived in a casemate directly over the Sally Port near the area where most of this activity took place. 

Hurricanes were a constant threat. One of them struck on October 22, 1865 while Dr. Mudd was imprisoned in the dungeon. Trees were uprooted. The cattle pen in the middle of the parade ground collapsed and the cattle roamed free in frightened confusion. With the wind still howling in the darkness before dawn the next morning, the upper story of the south section of the officers quarters blew out, killing Lieutenant John W. Stirling in his bed and injuring Captain R.A. Stearns. The walls and roofs of many other fort buildings were severely damaged. It would take months to repair the damage.

Summertime temperatures at Fort Jefferson hovered in the 90’s. Temperatures over 100 degrees were not uncommon. The high temperatures combined with the high humidity produced a broiling sensation that could not be escaped. The winter months provided some relief, but could still be uncomfortable. 

Before his attempted escape in September 1865 Dr. Mudd had been working in the prison hospital. After his attempted escape he was assigned to manual labor - sweeping down various parts of the fort. During the last two years of his imprisonment, Dr. Mudd worked in the carpentry shop with Edman Spangler, and also sometimes worked as a clerk in the Provost Marshall’s office.

Bad food and poor living conditions affected everyone’s health. Scurvy was a major medical problem at Fort Jefferson, resulting from the lack of fruits and vegetables containing vitamin C. Arnold and Dr. Mudd both suffered from rheumatism. Dr. Mudd lost weight – he normally weighed about 150 pounds. 

Many military posts, including Fort Jefferson, had convenience stores run by civilian businessmen known as sutlers. Soldiers and prisoners could both purchase items at the sutler's store. Prices were high and the quality low, but there was nowhere else to go. Sutlers sold clothing, ink, pencils, writing paper, preserves, tobacco, canned food, pins, mirrors, pocketknives, toothbrushes, hairbrushes, and a wide variety of other everyday items. Emily Holder, the wife of a Fort Jefferson assistant surgeon, wrote that she was able to buy a stove and other necessities for setting up housekeeping from the sutler’s store. Whiskey was also available from the sutler’s store, and drunkenness on the part of soldiers was a common disciplinary problem at the fort.

Prisoners also received food and other items in packages from home. Dr. Mudd and his fellow prisoners received clothing, canned fruit and vegetables, tobacco, and sometimes even whiskey, although the whiskey often mysteriously disappeared from the packages before being given to the prisoners.

Friends and family supported Dr. Mudd during his imprisonment by sending him letters and newspapers as well as spending money and food.  Two mail boats a week normally arrived at Fort Jefferson, but sometimes the boats were delayed due to quarantine procedures designed to protect against yellow fever and other diseases. Dr. Mudd’s mail usually encountered additional delays as it passed through the fort’s military censor. While the mail from family and friends was usually upbeat about his chances for release, the newspapers allowed him to form a more realistic view of the political climate and his likelihood of release.

Fort Jefferson also had a small library that both prisoners and soldiers could use. In an October 18, 1867 letter to his wife, Dr. Mudd said: “I have access to a very choice library of over 500 volumes.” Records show that the small library also subscribed to newspapers such as the New York Herald, the Washington Chronicle, and the London Illustrated News. 

There was obviously little opportunity for entertainment at a military prison located on a desert island. However, although neither Dr. Mudd nor Samuel Arnold mentioned it in their writings, the officers and civilians at the fort did organize plays for entertainment. Both soldiers and prisoners were able to attend.