Pantry moths require grains, pet food, rice, or other food items to feed on as larvae. If they are surviving long-term in a wall void, they must be spending time outside of the walls to find food. 

If you've seen any in the kitchen or pantry, take a close look at bulk items for evidence of larvae. Seal these up with clips or in airtight containers. Chemicals aren't usually recommended for killing pantry moths. 

They shouldn't cause damage if they are living within a wall or chimney. However, your control options will be limited. If you are able to use a shopvac near any gaps or at the bottom of the fireplace, that may help remove some material and/or insects.  If none of the above, work this might be a case best left for a pest control professional to solve.