Many factors may be causing this issue.  One is the tarnished plant bug.  Its feeding can lead to crumbling fruit.  Check your plants to see if this insect is present. Another possibility is gray mold, which spreads in rainy weather, such as we have been experiencing.   It's the most common disease affecting raspberries in Minnesota.  Read the attached links; the second one contains a section about this fungus an gives suggestions for how to counter it.  Good luck with your raspberries.

https://apps.extension.umn.edu/garden/diagnose/insect/garden/beetles/medium/tplantbug.html
https://extension.umn.edu/fruit/growing-raspberries-home-garden