Captain Salem Avery purchased the property in the 1860s. He was a prominent waterman/ oyster buy boat captain from Long Island, NY. He moved to Shady Side to work on the Chesapeake Bay, where oysters were more prosperous. During the warmer months, he was a farmer. He owned several acres of farmland throughout Shady Side, MD.
In 1923, the Avery family sold the house to the National Masonic Fishing Club —comprised largely of Jewish families. Anne Arundel County had segregated beaches, which prevented Jewish families from public recreation. Among the fishing club's members were the family of Samuel Gompers, founder of the American Federation of Labor and a national advocate for workers’ rights and social justice. For decades, Jewish families spent their summers in Shady Side, finding belonging, recreation, and refuge along the Chesapeake Bay.
Desegregation gave less of a need for the Jewish Fishing Club in Shady Side. The Bay Bridge allowed vacationers to access larger beaches, resulting in them spending summers further away.
In 1989, the Jewish Families sold the property to the Shady Side Rural Heritage Association. The purpose was to turn the building into a museum that honored local history, watermen culture, and have a positive environmental impact.
Today, we are a museum that provides educational outreach to the public, children's activities on Saturdays, and environmental stewardship.