You are correct--it is Asian bush honeysuckle and it should be removed.
I purchased this at a native plant sale as Diervilla lonicera. Found the tag while digging it out. Probably still parts left that will come up next year so I want to ask: Are you sure it is the asian?
You can take a sample to your local Penn State Extension and ask for it to be identified.