bees building hive in house foundation blocks
Most likely what they are are carpenter bees. The female will make the nest in the spring and the new adults will emerge late summer. I would wait until the adults emerge and then repair and paint the window frame. This will deter them from wanting to use the nest again. Here is a good article that includes information about their life cycle and prevention: https://extension.umd.edu/hgic/topics/carpenter-bees
My bees don't match the description of the carpenter bees. They are smaller, slimmer and lighter-colored.
Does the mitigation strategy change if they are honey bees or some other kind of bee?
I doubt they are honeybees - when people get a hive in their homes, it's usually in a wall cavity or eaves. Carpenter bees are the usual suspects for living in spots like this, but not the only type of bee that nests this way, so I think that that the same strategy should work.