Holes in river birch leaves
Several insects will feed on birch leaves. Sawfly and birch leafminer are examples. Your picture appears to show some birch leafminer damage. The black frass is likely ‘bug poop’ as you said. These are cosmetic and don’t need treatment unless the tree is otherwise stressed by bronze birch borer in the trunk and branches, drought, or is otherwise unhealthy.
The best thing you can do for your trees is to keep them watered during these hot dry times. Soil should be moist at least 6-12 inches down after watering. Removing grass in a circle around the tree and mulch,, without piling mulch against the trunk, is helpful and removes competition for water. River birch, as the name implies, prefers constant well-drained moist soil..
References for leaf feeding insects on birch-