Thanks for including the pictures.  You're actually not dealing with cedar-hawthorn rust, even though that would make sense with your hawthorn being infected.  Your tree is suffering from cedar-quince rust. 

There are several rust fungi that alternate back and forth from a juniper like eastern red cedar (Juniperus virginiana) and a member of the rose family (Rosaceae) like hawthorn, quince, apple, pear, serviceberry, etc.).  So, except for the cedar part (which is a juniper!), the names don't always reflect the rose host.  For example, cedar-quince rust also infects all of the members of the rose family that I listed in the last sentence, but it's just named for one host.

The same is true of cedar-hawthorn rust, but the symptoms are much less damaging which is why I'm risking creating some confusion by explaining exactly what's going on.  If you look into "cedar-hawthorn" rust, you'll find it only infects the leaves of hawthorn. So, it's not as damaging.  But, if you look into cedar-quince rust, you find it not only infects the fruit (= haws) of hawthorn, which is not very damaging, but it also infects the stems which is very damaging and is what you're seeing on your hawthorn.

Unfortunately, environmental conditions this spring created a perfect storm for infections to hawthorns and other rose hosts by the cedar-quince rust fungus.  You can see this on other hawthorns as well as Callery pears with the fruit sprouting the orange spore structures like you're seeing on your tree.  Of course, the spore structures on your tree appear to be sprouting from stem cankers which is the most damaging symptom of the fungus.

I appreciate you including the pictures because they tell me that your tree is suffering from the most severe form of this disease and the cankers commonly lead to dieback of the infected stem.  So, the disease will affect the overall growth and "shape" of your tree.  There's another challenge that you may also be dealing with:  all hawthorns aren't the same in terms of susceptibility to the disease.  It appears that you have a very susceptible cultivar.

Despite claims otherwise, need oil is not a good fungicide.  But, even the best fungicides often fail to suppress the disease on a highly susceptible hawthorn if environmental conditions support heavy infections and disease development.  Plus, you would have needed to make multiple applications starting while the tree was flowering.  So, probably wouldn't have been able to spray your way out of the problem no matter what you did.

Pruning out the cankers also provides limited control because you don't see the cankers until the fungal infections are well on their way.  And, reducing the spores coming from the orange fruiting structures only protects junipers from infections.  The fungus must alternate back and forth.  Spores from the hawthorn can't infect members of the rose family, only junipers, and spores from junipers can't infect junipers, only members of the rost family.

With all of that said, you have a couple of options.  It's still a good idea to prune out the infections to help break the cycle involving junipers; however, I'd say there have already been a lot of spores produced.  But, you'll need to see if the pruning destroys the shape of the tree.  If not, you could also hope that environmental conditions aren't the same next season and you won't see such heavy infections.  Of course, none of that changes the problem with your hawthorn being very susceptible to the disease, so it's likely you'll see it again.

Before I looked at them, I would have said that cedar-quince rust