You should keep the plastic wrap over the cartons until you see the seeds sprouting. Once the tiny plants come up, you can remove the plastic. However, make sure they do not dry out. While planting in egg cartons can be a great choice, the cardboard can wick moisture away from the little plants so be careful to not let the soil get very dry. You may need to water daily yet the soil should not always be soggy wet either. Soil needs space for air and water for the roots of the plants. If you plant the seedlings in the ground directly, remove any cardboard that would be above the ground as it will again wick away moisture from the plants. Here is further information on growing herbs from seed.

https://extension.illinois.edu/news-releases/start-herb-seeds-indoors