This is a type of solitary wasp, and unlike social wasps (yellowjackets, paper wasps, and hornets), they are not inclined to sting since they do not share a nest with their siblings and a queen that need defending. A female solitary wasp could sting if she were needing to defend herself (if stepped on, for example or swatted and caught by hand), but otherwise they will ignore people in favor of working to catch insects to stash in her burrow. She will not raise her young, but provides them with insects to eat and then seals-up the burrow and either creates one or two more or dies at the end of her life cycle. Male wasps cannot sting (though they often look nearly identical to the females), but they aren't going to be digging a burrow or bringing insects to it.

It's hard to ID this wasp to an exact species, as there are several look-alikes in Maryland that are blue-black with iridescent blue wings. However, all hunt insects and can visit flowers to drink nectar. Which insects they hunt will depend on the species of wasp, but can include crickets, caterpillars, and even spiders.

Activity might last another couple of weeks (hard to say, as it depends on when it began, and which wasp this is), but then will be finished for the year. Treatment with a pesticide is not needed or recommended, as they are valuable members of our local ecosystem. If absolutely necessary due to sting allergy risk or aggressive behavior on the part of the wasp (not very likely), then you could treat the burrow entrance with wasp spray. Follow all directions on the product label. Commercial insecticide products labeled for use on wasps are the only option; home remedies using alternative or household ingredients will not only probably not work, but they are ill-advised for a range of reasons.

Miri

We understand what you are seeing.
They are termed 'solitary' because each individual female makes a hole and uses her stinger to provision it with a paralyzed spider or caterpillar along with her egg, for her hatchling to eat. (The males may fly around but have no stinger and the females are only interested in getting her work done.)
When conditions are ideal- dry, loose soil without any covering, several will often site themselves nearby, but still only one per hole, so 'solitary'. 
The most important thing to remember is that solitary wasps are beneficial pollinator insects that we encourage conserving. They are not around for the whole season, and you can walk amongst them, and they will not bother you (unless you squeeze one, perhaps.)
On the other hand, what we term "social" wasps, like yellow jackets, live in a large, growing community hive. They defend that communal nest aggressively when disturbed, tend to be angry and can sting repeatedly. 
Here is our page where you can learn more: 
https://extension.umd.edu/resource/predatory-wasps/

Generally, if you can grow a groundcover or keep the soil moist/sprinkled or mulched earlier in the season they may decide to go elsewhere in the future.


Christine