Flower traits are a key feature for plant ID since foliage traits are more variable, but based on the foliage and branches pictured, this reminds us of Bayberry, also known as Waxmyrtle (genus Myrica, sometimes listed as Morella). Two species are available at nurseries, both native in Maryland. Bayberry foliage is very aromatic when leaves are rubbed or crushed...does this plant's foliage have a scent? (It's somewhat spicy-like, but hard to describe.) Bayberry does technically flower, but the blooms are not showy; the berries (on female shrubs that are pollinated, at least) are one of the features the plant is grown for, either for decoration (they're pretty small, though), wildlife (birds love them), or for craft (making bayberry-scented wax candles). Sometimes bayberry is evergreen and sometimes it isn't (the latter appears to be the case here, if the ID is correct). Mature size varies considerably, ranging from around 5 feet high and wide or so (unless it's a dwarf cultivar) to more than twice that size.

If the plant's foliage is not scented, you could wait to see if it blooms (either here or moved to another planting spot if it's in the way) at some point later this year or next spring.

Miri

Okay, that supports our guess of Bayberry, then. They are nice native shrubs, so if you wanted to keep it, replant in a location that receives full sun (they can tolerate semi-shade, but may stay sparse) and where the soil drains well. They can also tolerate temporary soil wetness, but in general tend to grow best in sites that aren't soggy for very long.

Miri