Although it is difficult to tell for sure without seeing a sample of your plants or a picture, I think some spotting with a rusty coloration on toad lily might be a fungal leaf spot called anthracnose. Toad lily or Tricyrtis does not get a fungal rust infection, which would be characterized by pustules which break the surface of the leaf and have rusty or brown spores present. You would be welcome to send in a sample to your closest Cooperative Extension office (Carvel Center on Rt 9 in Georgetown).
I would recommend trimming back the foliage this fall, or in the very early spring, as fungi can survive on small bits of plant debris on top of the soil. The fungi are natural decomposers in our ecosystem. Next year, when the foliage comes up, I would recommend putting it on your calendar for July, to spray with a fungicide. There are several fungicides that may be available in the garden centers and stores for home use, but please check the labels. Remember that most fungicides are preventative only, and need to be re-applied in about 10 days to protect against fungal infection.
Thank you for contacting Cooperative Extension,