Thanks for contacting Ask an Expert. Without seeing a picture of the insect, I cannot be sure it is a wasp. However, I can tell you a few steps to rid your walls of the wasps. First, you can leave them there. The new queens will emerge in a few weeks and leave and the entire colony will die. If you know where they are coming in you can use the spray can of insulation you buy at hardware stores to fill cracks and stop up the entrance after you put in some steel wool. If you do not know where they are getting in, put hornet nest traps outside near where you have seen them. You can use the pheromone the traps that come with the traps, but I would put a small piece of raw meat in the interior. This will draw many more wasps.
You mention nests plural. You may need a carpenter to open the wall where the nests are and have a pesticide person there to spray them.
If you have time, please let me know how many nests you think you have, where they are getting in or coming out, and typically what type of interference are they - outside or inside, a few yellow jackets or a lot, and are you sure it is a wasp. Thanks
Glad it worked. Almost all wasps seem to be able to eat through some materials, and that includes the foam I suggested, however, I have never had any of them eat through steel wool, including mice.
I think you are smart to wait and watch then seal everything in the cool autumn, when most if not all will be gone or dead.
Let us know if there is anything else we can do for you. I am going to close this inquiry.