It's hard to tell without being able to see any symptoms on the fallen (or pre-fallen) fruits, but possible reasons for premature fruit shed include drought stress (both years were/are in drought, unless plants were being watered regularly), poor pollination (if the spring bloom window was too cold or wet for enough pollinators to be out), and fruit damage by the typical pests like Plum Curculio and/or Oriental Fruit Moth. These pests are usually suppressed by routine pesticide applications, but if trees are not on a spray schedule, that might explain why an outbreak caused significant fruit loss.

Miri