Those are the overwinter cases of bagworms. There are eggs inside. You can cut the silk that was used to attach them to the plant and then destroy them. When they hatch, they are little caterpillars that feed on whatever plant they hatch on. In your case, they will cause more damage to the plant. I can't tell if there is anything else going on with the arborvitae, though the vinca growing around the base of the plant can be a competitor for water and nutrients that doesn't benefit the arborvitae.
Should I spray the other arbovitae so the caterpillars do not spread? What should be used?
If you are confident you can pull/cut the bags off of the plant now, you do not need to spray. If you want to spray, the eggs typically hatch by mid-June. This can fluctuate depending on temperature. The early stage caterpillars are susceptible to products with the active ingredient BT (Bacillus thuringiensis) or spinosad. These have little to no impact on other insects and are very safe to use.