What plant is this?
Your weed is commonly known as 'Pokeweed'. All parts of the plant are poisonous if ingested, although birds will apparently eat the berries and deposit the seeds far and wide. It is appropriate to pull up the young plants when first observed and before they produce fruit. You will notice that the plant has a deep, fleshy root and when the plant is fully grown, you may have to dig out the root in order to get all of it.