While the house was under construction Andrew Low's wife, his four year old son, and his uncle died. The uncle willed his entire estate to Andrew, so it was a somber but wealthy Andrew Low II and two young daughters who moved into the fine house at the close of 1849.

Five years later, "bright, gay, beautiful" Mary Cowper Stiles, daughter of William Henry Stiles, United States Minister to Austria, stepped through the classical entrance as the bride of Andrew Low. In time, three little girls and a son and heir, William Mackay Low, brought new life and activity into the house.

The children's bedroom

Many splendid entertainments took place in the parlors. William Makepeace Thackeray was twice a guest in the house while on lecture tours in the United States in 1853 and 1856. The desk at which he wrote when he was the Low’s guest may be seen in one of the bedrooms.

Andrew Low’s most famous guest was General Robert E. Lee, whose friendship dated back to West Point, where he roomed with Mary Low's uncle, Jack Mackay, of Savannah. Lee was a frequent guest at the Mackay's Savannah home while posted at Fort Pulaski.

Lee had also courted Mary Low's mother, Eliza Mackay. Upon Miss Mackay's marriage to William Henry Stiles, Lee gave her a gold brooch set with seed pearls and garnets. Thirty years later, Lee was godfather to Mary and Andrew Low's daughter, Jessie and on his last visit to Savannah in April 1870 General Lee was Andrew Low's house guest for a week.

The city was under military occupation, so no public ceremony could be held. Citizens came to the door asking to "shake the General's hand" and a reception was held in the double parlors in his honor attended by the Mayor, the Aldermen, and many prominent citizens.

Eliza Stiles wrote of the New Year's Eve reception in 1866 stating that the girls "dressed the house with flowers" (probably camellias) and "30 gentlemen called." For large entertainments the sliding doors between the parlors were opened and the furniture pushed against the walls or removed.

The Low's fondness for dogs is reflected in the needlepoint fire screen in the family parlor. The pattern depicts the likenesses of two of the many dogs owned by the Lows. The frame of the fire screen was saved from the home of Mary Low's grandfather, Joseph Stiles, which was torched by Sherman's troops as they marched toward the city.

In 1886 William Mackay Low inherited his father's vast estate in both England and America. Six months after the death of William's father, he married Juliette Gordon who later founded the Girl Scouts of America.   Daisy, as she was known, visited her future husband's father in England and became great friends with William's sisters. It was over there in England that she and William fell in love. The couple spent much of their time in England using the Savannah house mainly as a winter residence.   After William's death in 1905, Juliette went back and forth to England many times. The mansion remained in the Low family until the death of Juliette Gordon Low in 1927. A year later the house was bought by The National Society of The Colonial Dames of America in the State of Georgia to be used as its state headquarters.   It has been open to the public as a museum since 1952.