While I am not 100% certain on this weed, it appears to be field horsetail (Equistetum arvense L.), a branchless plant that looks like a bottle brush. While its fertile cones produce thousands of spores in early to mid-spring, it reproduces mainly by rhizomes that can grow to about 2 yards.

Rhizomes are difficult to eradicate because they break easily when cultivated, leaving small remnants beneath soil that can sprout anew. One non-chemical method, if it is possible in your situation, is to mow the area repeatedly, leaving little opportunity for the roots to regrow and supply nutrients to the plant. It will take some persistence to starve the rhizomes, but it is possible. It won't be quick but is a viable non-chemical response. 

While herbicides are generally ineffective, look for products that are  effective at killing roots. It will require persistent treatment both before and after the growing season for several years to get good result, but considering what's likely without treatment, it's worth a try.  

I am not permitted to recommend any specific herbicide, but if and when you decide to use synthetic chemicals, be sure to read the label carefully. Make sure what you choose has the plant characteristic you wish to target -- roots and rhizomes. Follow the directions exactly; the label is the law. You may want to contact your county extension office for more guidance. The website is cuyahoga.osu.edu
 
Reference: Uva R., Neal, J., DiTomaso. J. 1997. Weeds of the Northeast. Ithaca and London. Cornell University Press.
https://crops.extension.iastate.edu/encyclopedia/equisetum-biology-and-management