I stored leaves from last fall in garbage bags over the winter. I intended to take them to the recycling center. (STUPID, I know). Unfortunately, I am just now trying to dispose of them. It did not take long to discover yellow jackets in some of the bags. I sprayed! I still have them in more bags. Other than sprays or an exterminator, do I have to wait for a killing frost.
Your safest option is to wait for the killing frost.
Don't berate yourself for storing leaves over winter. Many people do that for a different reason. Those dry leaves are an excellent source of carbon to be mixed with grass clippings and other green materials and kitchen scraps in order to produce compost. That can still be done once the yellow jackets are gone.