This is likely caused by the dreaded codling moth.
Codling moths, Cydia pomonella, are a serious pest of many tree fruits, including nectarines. Codling moth larvae can enter fruit at the stem end, sides, or calyx end, and can cause severe damage by feeding around the pit. A characteristic sign of infestation is one or more holes on the fruit's surface plugged with frass. Here are some ways to manage codling moths:

• Sanitation Check fruit for damage every week or two, starting about six to eight weeks after bloom, and remove and destroy any infested fruit. This can be an effective method to reduce the population before the larvae can crawl out and start the next generation. You can also pick up and throw away fallen fruit in a plastic bag each week.
• Trapping Set traps in mid-May to check for codling moths in your area. Clean out the traps each time you check them, and use tweezers or disposable gloves to handle the lures to avoid transferring pheromones to other surfaces. If you don't catch any adults, you might not need to use pesticides.
• Physical controls In June and August, attach corrugated cardboard strips to the tree trunk and scaffold branches to provide a place for larvae to make cocoons. The strips should be at least a foot and a half above the ground and have vertical corrugations. After the cocoons form, remove and destroy the strips.
• Parasites can attack codling moth eggs and larvae.