Japanese beetles have just shown up in my yard this week.

This is definitely insect damage, possibly caterpillar of some sort.  Without a picture, its hard to identify the culprit.  Caterpillars are sometimes very well camouflaged and easy to miss.      There is no permanent damage, just unsightly.

Japanese beetles are starting to emerge now, but based on your description, you probably would have seen them at the times of day you were looking. Another common insect are earwigs. They are flat, red-brown insects that are about 5/8 inch long. They are notable for their cerci -- long pincer-like structures at the tail end of their body.  Earwigs are usually found under objects or in natural settings, under bark. They are nocturnal scavengers but will attack some plant species by chewing irregularly shaped holes in the petals and leaves.

Without being able to confirm what insect, it is difficult to suggest an appropriate treatment/deterent.  But here is a good general guideline: https://extension.colostate.edu/topic-areas/insects/insect-control-soaps-and-detergents-5-547
and
https://extension.colostate.edu/topic-areas/insects/european-earwigs-5-533/