Thanks for your patience with my response time because I consulted with an expert here at UVM before responding to ensure I offered the best info that I could… Essentially the consensus given your information and photos is that you are dealing with two separate diseases, black knot AND brown rot. We can see black knot on the branch and brown rot given the brown spores on the fruit.

Black Knot 

Info: https://extension.umn.edu/plant-diseases/black-knot

Pruning can help but will not cure. Fungicides are not very effective and they would have to be sprayed weekly. Plums can remain productive with this disease for many years. For more info, check out section, "How to manage black knot”, in the article linked above.

Brown Rot

This was a big year for brown rot, we’ve seen a lot of it around Vermont!
Info: https://hort.extension.wisc.edu/articles/brown-rot/

Infections occur when we have wet weather at bloom which can cause a shoot blight but then infects the young fruit that develop the brown spores on the fruit. Pick and destroy infected fruit so it does not overwinter. You can spray fungicides at bloom but it is typically not a problem every year due to weather.

Brown rot is not a lethal disease, but once fruits are infected, there are no curative treatments. To manage twig infections, prune four to six inches below sunken or dead tissue on each branch. Dispose of these branches by burning (where allowed by local ordinance) or burying them. To prevent spread of brown rot fungi on pruning tools, decontaminate tools between each cut by treating them for at least 30 seconds in a 10% bleach solution or preferably (due to its less corrosive properties) 70% alcohol. Rubbing alcohol and many spray disinfectants contain approximately 70% alcohol and are easy to use. If you use bleach, be sure to thoroughly rinse and oil tools after pruning to prevent rusting.

Remove and destroy any wild or volunteer Prunus trees and shrubs on your property, as well as all rotting and mummified Prunus fruits, as these can be reservoirs for brown rot fungi. Burn (where allowed by local ordinance) or deep bury these materials. Thin your Prunus trees to increase air flow and promote more rapid drying of twigs and fruits. For information on how and when to prune, see UW Bulletin A3629, Growing Apricots, Cherries, Peaches, and Plums in Wisconsin (available at https://learningstore.extension.wisc.edu/). As I said above, be sure to decontaminate cutting tools after tree/shrub removal and pruning. Carefully handle fruits during harvest to minimize bruising, and store fruits in a cool, not overly wet environment.

I hope this is helpful and good luck with your remaining harvest!

Thank you!  Very helpful!