Eat the slick interior of an oyster and you’re left with two shells, layered like mica or phyllo, blues shading into grays shading into lavenders and creams. I’ve never found a pearl in one, though I think of Steinbeck’s The Pearl (first read in fourth grade) nearly every time I eat an oyster.

But today it’s the debris that interests me, those massive piles of shells left after the sweet bivalve has been plucked. And particularly the strong-brittle substance known as tabby, which, like so much else, enslaved African Americans learned to fabricate and then turn into objects of utility: oyster-shell bricks with which to build houses, their own and those of the people who professed to own them. People like my ancestors.

I have such a brick in my family room here in Ann Arbor. It came to me from my late mother, who undoubtedly explained its origins to me, but I’ve forgotten them. I assume it’s from coastal Georgia, where my forebears grew cotton. The land is saltwater marsh, a fragile ecosystem then and now, riddled with snakes, gators, crabs and other crustaceans. In the summer, 19^th-century planters used to escape the malarial swamplands by heading inland. They left their slaves to contend with the pests and keep working the land—dredging canals, clearing tick-ridden woods to make fields, hauling supplies from wharf to plantation to wharf, endless labor. And of course picking, sorting, baling cotton. Men and women both were expected to pick 80, 90, 100 or more pounds of cotton a day; a shortfall meant whipping.

A man I met last year at a conference, a former staff member at the Historic Preservation Trust, has proposed a “Who Built America” campaign to mark—and celebrate—the anonymous contributions of enslaved African Americans to the nation’s infrastructure. To date he’s got no takers.

Many of these things came to mind in May as I stood deep inside a grove of tick-ridden woods in coastal Georgia, on acreage my family owned in the years leading up to the Civil War. I’d gone in with an archaeologist who’s been working on a dig to retrieve artifacts from a  a half-dozen former slave cabins on the site. The foundations of those cabins are tabby brick—so much tabby, in fact, that the archaeologists are using some of the bricks to hold down the protective tarps they lay over the sites each evening. Together, the archaeologist and I inspected two of the cabins where she and her team have found the remains of adjoining tabby fireplaces. On either side of a shared chimney, a family of as many as ten would have lived. The space is maybe twelve square feet. It was humid and buggy the day we went in, and when we emerged, we each picked nearly a score of tiny red ticks from our pants and shoes.

Inside one cabin, the team had found a nineteenth-century penny with the word “Liberty” stamped on its surface. Perhaps the enslaved people who lived here studied it with hope? More likely, they doused their fears and sorrows with one of the multiple pipe bowls the archaeologists have also found at the site, along with shards of pottery and glass, all of which will eventually be taken to an archive for storage.

But not the fireplaces. Not the tabby foundations of the cabins. Those will be turned back into the earth and ultimately covered with an asphalt parking lot. My first—and enduring—impulse is to rail against this desecration. Americans are quick to fault others (the Taliban, ISIS) for destroying cultural heritage in the name of religious ideology. But we do it in the name of commerce.

Some small part of me resists, however. Maybe it’s fitting that this awful part of American life—what journalist Charles P. Pierce calls “the mother of all American crimes”—be pushed back into the soil and buried in our humic earth. Maybe it’s fitting that these countless bits of oyster shell—these tabby bricks, molded by nameless, countless hands—be restored to the Edenic place from which they came, a world not yet corrupted by human greed.