This looks to be a native annual plant, commonly called "touch-me-not" because the seed capsules seem to explode and scatter the seeds. It is in the genus Impatiens.

Here are a few options:

Impatiens pallida (yellow flowers).

Impatiens capensis (orange flowers). Often called jewel weed.

Here's an article from Wisconsin on them with good photos.

I doubt that it is the same annual impatiens that you can buy as a shade plant in nurseries in the spring. Although they do seed themselves, I think the cold winters where you are located would kill the seeds.

If I am correct that these are the native impatiens, you can simply pull them and discard them if they are too much. You do not need to spray an herbicide. They are shallow rooted and easy to pull. If you don't like them, be sure to remove them before they go to seed. They will die with the first frost.

You might consider leaving them at least until they bloom. Both hummingbirds and our native bumble bees use the flower for nectar. Some people claim that crushing the leaves on your skin relieves poison ivy itch, but there aren't scientific studies to back up this claim.