This is a common question we receive this time of year. The answer is complex, but in short, it is likely a combination of temporary seasonal fluctuations as well as chronic long-term issues. August tends to be the peak of adult butterfly abundance (or at least apparent abundance) in our area, especially among the easy-to-spot swallowtails which congregate on their favorite perennials of Joe-Pye Weed, Common Milkweed, Blazing-star, Monarda, and Ironweed. Joe-Pye Weed is not in flower this early, so when it blooms in August and September, the butterflies will probably congregate on it.
There could be many other reasons (perhaps multiple reasons at once) that are causing an apparent decline in the local butterfly population.
loss of habitat - while your yard flora is probably stable, public-land and private lots are not, alas, given continued development and the replacement of lost plants with non-natives (often) or plants which happen not to support the butterfly species in that area; most species are quite host-specific and have a relatively small range of caterpillar food options. Invasive plant species also displace native host plants, and deer (which avoid most invasive species) have become overpopulated and are consuming proportionately more native species than they used to. While they cannot reach mature trees (a food source for multiple butterflies), they can stunt or kill saplings that are needed to replace trees dying of natural causes in the woods.
use of pesticides elsewhere - we do not have precise data on how far various butterfly species fly and disperse, but blanket sprays for other pests (mosquitoes, Gypsy moths, etc.) can impact butterflies if the pesticide is not chosen wisely (or it's not possible to eliminate non-target impacts like this)
predator/parasite population dynamics - prey species and their predators (or parasites) can have "boom" and "bust" years, and tend to be cyclical, the peaks chasing each other as one population rises and then the other; winter and spring weather may play a role in survival of both species as well, possibly tipping the balance any given year
loss of host plants - as one example, the invasive pest Emerald Ash Borer has decimated our Ash tree populations, and Tiger Swallowtail uses Ash as one of its caterpillar host plants. We do not know if Ash plays a critical role in their life cycle (as they can use other species as well), or if Ash conveys any improved survivorship of caterpillars for some reason, but it is an instance of a supporting host plant no longer being widely available to a species. In addition, as mentioned above, several canopy trees play host to multiple butterfly species. Home landscapes, neighborhood street trees, commercial property grounds and so forth don't always plant these large species, instead opting for smaller and/or more decorative flowering trees that don't support the same (or any) butterfly larvae. For example, Hackberry trees support several butterfly caterpillars, but because Hackberry happens to have a horrible public-relations name and no particularly showy flowers or fall foliage color, it isn't planted as often as, say, the smaller and colorful Crepemyrtle, which supports none of our butterflies.
insufficient nectar sources - populations of some native "weeds" that grow primarily in disturbed sites, like Common Milkweed, can be mowed-down by maintenance crews when growing by roadsides (as presumably they are required to keep that shoulder buffer accessible) and in other unkempt wild areas near developed lots. Even though adult butterflies may get by using other nectar sources (Butterfly Bush, etc), perhaps these sources are not providing the same nutrition and therefore the reproductive health of the population is compromised.
timing - although butterflies tend to have multiple generations per year, the periods when most are in their non-adult life stages cause a lull in adult butterfly sightings; overlapping generations can occur, but as we approach late summer, more of the species that overwinter as larvae or pupae are prevalent as adults because they are in their final throes of breeding for the year. This tricks us into assuming butterflies are lacking in mid-summer when they may not be. In addition, this year's late cold snaps could have set-back the first generation of early-emerging butterflies.
- population decline - sometimes it's just because populations are really on the decline overall, and this is blamed on a number of co-occurring factors which we touched-upon above: primarily habitat loss, pesticide use, and freak weather events. How much climate change is involved is still being studied. Generalized insect surveys (and even anecdotal observations) in various countries have noted an apparent substantial decline in insect populations overall.