The damage is caused by the four-lined plant bug.
While their damage is unsightly, it is not life-threatening to affected plants. One of the easiest ways to deal with the damage on herbs and perennials is to wait until the insects are done for the year and then cut the plants below the damage. They will grow back nicely with no more damage. While this can delay blooming of perennial flowers a bit, removing the affecting foliage improves the appearance of severely damaged plants.
If the damage is too bothersome and you prefer to spray, begin applications at the first sign of their activity because just a few bugs can create a lot of damage. And nymphs are easier to control than adults. Insecticides labeled to control four-lined plant bugs include insecticidal soap (nymphs only), neem oil, pyrethrins and piperonyl butoxide and carbaryl (Sevin).