Perilla is an alien invasive plant in Maryland. (It's of Asian derivation.) The weedy perilla is usually green, but because there are purple ornamental varieties on the market, you may be seeing a hybrid. People also raise it as an herb.
At any rate, we'd recommend that you pull them all before they can go to seed.