The large wasp is indeed a European Hornet, and the smaller wasp does look like a Yellowjacket (though we can't tell which species of the several that occur in Maryland). Both are likely queens, as it's too early in the season for any of our social wasps to have enough workers in a nest to be encountered. No social wasp species in Maryland re-uses nests (it would be very rare for a nest, even sheltered in a building wall, to survive the winter), but queens are out now exploring sites for starting their own new nest. Therefore, isolated encounters like these are likely just due to chance as a female happened to find her way inside as she explored. We can't tell if either female overwintered inside and is just now trying to leave, or if a wasp from outdoors got in; either scenario is possible.

Sealing gaps/cracks or vent cover access points in exterior walls can be an easy way to discourage them from getting in. Similarly, worn-down door/window weather-stripping can be an entry point, as would be any torn window insect screening. European Hornets fly at night as well as during the day, so any areas with night lights outside a door or window might be drawing them in, so consider turning those off in that case. More information about the habits of these wasps can be found on the linked page above.

Miri