flowering Dog Wood
There could be many reasons that your dogwood could be having trouble. It certainly could be an insect pest (like borers) or diseases, but it also could be an unrelated environmental stressor like drought or dry soil in blazing sun.
Have you noticed any pattern to the blooming? Like a heavier bloom set every other year?
Watch your tree leaf out and grow this season and see if the leaves are of normal size and color, and that branch tips put on some good new growth.
Take a look at this IPM:Dogwood publication that lists they many problems these trees can develop in our area, and what you might do about them: