Did you consult with a certified arborist or a licensed tree expert? 

We don't think that it looks like borer holes based on these photos. Possibly old Sapsucker damage, which is from a type of Woodpecker that overwinters in most of our state. Typically, they do not cause enough damage on their own to kill the tree unless it is stressed for other reasons. 

Removing the dead wood is helpful, but if borers are present in the wood, there is nothing that can be done at that point. Borers are typically a secondary health issue as well so if the tree is stressed from woodpeckers, along with environmental conditions, it could be what caused the borers. 

While silver maples are fairly tolerant of various conditions, if you have had brackish or salt water flooding for periods of time, it could be causing decline as well. They don't typically grow well in high-salinity soils.

Emily