I have Carpenter bees perforating my barn beams. I saw them what appeared to...
Rather than try to summarize or cut and paste, I will refer you to the last three sections of the OSU Extension Factsheet on carpenter bees: https://ohioline.osu.edu/factsheet/hyg-2074. That should answer your questions and provide options.
Good luck on quickly minimizing the damage.