This is a yellowjacket; several species of yellowjackets occur in Maryland. You can learn about their basic life cycle and habits in the linked page about social wasps. They are likely queens, which are the only individuals from last year's nests that remain alive over the winter. Queens seek shelter in areas that protect them from the brunt of the cold, and sometimes that means they find their way into a building, where they essentially hibernate until spring. They will try to escape being trapped inside by flying to a light source (a window or lamp), because out in the wild, heading towards the sky would be how they get out of a rock crevice, tree hollow, or any other place they've tucked-into for the winter. The fact that you found several probably just means that several queens found a way into the cozy home for the winter and are now "awake" and ready to leave to find a nest site of their own.

It's too early in the season for an established nest to be present that has multiple worker wasps in it, so there is no still-in-use nest you need to worry about finding and removing. For now, just catch the wasps in a cup or some other convenient container and let them free outside. If you notice any access points attracting wasp attention outside, like gaps around vent covers, utility or pipe entry points, worn-down door or window weather-stripping, or any other holes or cracks, then they can be sealed with caulk to keep wasps and other nuisance insects out.

Miri

You're welcome!