A weed is just a plant in the wrong place! 

This plant is commonly called "ground cherry". The berries are edible, but the leaves, etc, are not edible . Some people encourage it and love it, other's pull it out and discard it. If you have livestock or other animals who might eat the leaves, you may want to pull and discard it. In other words, it's up to you to decide if this is a weed or not.

I am not sure which species you have, but I suspect it is Physalis longifolia or Physalis virginiana. 

Here's a link to more information on Physalis longifolia.

Be aware that leaves and unripe fruit can be toxic if ingested. On the other hand, you can buy the ripe fruit at a farmer's market. It is your call if you want to risk eating the berries. 

A google search turns up an interesting link between ground cherries and cancer prevention!