Hope you already know about Marshall's & TJMaxx for your garden. Granite gravel, #89, along with other bulk materials is available from Stone Forest.
For plants, locally, try: Buck Jones, Land Arts, Wilkerson Mill, Piccadilly, Goodness Grows, Ashe-Simpson, Pikes, Hastings, Habersham Gardens, & the ubiquitous big boxes.
Good pots, don't buy any other kind, are a specialty at Four Seasons Pottery.
Garage sales & rescues are necessary to every good landscape. Scott Antique Market, 2nd weekend each month, helps you create magic in your garden.
Nursery, 4 pics above, I shot in England at an estate garden open to the public. A fantasy. Where we all want to shop for plants, service, accouterments. Instead, gardens come together via hunting & gathering. Much hunting & gathering.
It's odd, gardening is a multi-billion/yr industry yet so difficult to shop for.
Garden & Be Well, XO Tara