Without seeing the whole bush, and by your description, it looks in pretty good shape. The lower leaves on these shrubs often get chewed by insects, but this doesn't affect the blooms at the top. Butterfly bush has very few pest or disease problems that require management. Since the yellow leaves are in the center of the plant, I can suggest a few possible reasons.

The most common problem is spider mites. They are very tiny. You can see them on the underside of the leaves using a magnifying glass. You can also shake a bush branch over a sheet of white paper and look for any moving specks. Spider mites draw out the sap from the leaves, leaving a discolored, yellow, or gray-mottled appearance. These mites show up when the conditions are dry or the plant is stressed somehow. Several other sucking insects might plague the plant, but spider mites are the most common. Look for webs in among the fading leaves. This will be the clue that insects are the culprits. Spray the top and underside of the leaves with insecticidal soap, or spray the top and undersides with a strong jet of water. This will knock off the spider mites. Avoid pesticides, as Buddleia attracts many beneficialinsects that could be killed. Do not spray when bees are active, and avoid spraying the blooms.

Your pictures also show some chewing damage, which can be caused by a large number of different insects. Water the bush deeply and regularly, but do not soak the roots.

Keep any yellowed or severely damaged leaves clipped off, leaving the main stem behind. Some insects are attracted to damaged, yellowing leaves. Depending on the plant's age, it may be time to think about pruning it in the fall. If you see small, dead stems in the center of the plant, you can clip them off now with the yellow leaves.

If you don't spot any spider mites or other insects, it's time to consider cultural issues with the plant. Buddleia prefers well-drained soil and full sun for best flower production. Wet roots can cause the plant to decline, and the roots may rot in overly soggy conditions. Soil pH is vital to plant health and helps uptake nutrients. Butterfly bush should be grown at a pH of 6.0 to 7.0. If soil is overly acidic, phosphorus ions react with aluminum and iron to form less soluble compounds. That means that those micro-nutrients are not readily available to the plant. If iron availability is low, the leaves will fade to yellow, leaving the foliage's green veins intact. When butterfly bush leaves are yellow with green veins, this is a sign of iron chlorosis and can be treated by sweetening the soil with lime. If you did not fertilize the shrub this spring, give a half or one-third dose of fertilizer now. A fertilizer listed for flowering shrubs is the type you want. Then return to fertilizing in spring only, next season.

I hope these suggestions help determine what is causing the plant leaves to turn yellow. Since it is primarily in the center of the plant, you should be able to bring it back to normal with these suggestions. Please do not hesitate to write back if you have further questions.