This looks normal for the older or most shaded leaves to color and drop off. Even an evergreen plant drops some leaves. Disease issues, like fungal leaf spots, would show up on a few green leaves too, and I don't see any of those infected. You can just remove the yellow ones, or leave them to fall off.
The plant looks recently cut back. Spring flowering shrubs are best pruned just after flowering. I honestly can't tell which Viburnum you have, though. Does it bloom fairly well, and when? OSU Landscape Plants lists pages of them, here.
This page from Penn State Extension gives more details on flowering shrubs. Those that bloom in summer or fall are often best pruned in winter. It also mentions "renovation" pruning, which is a technique that might help you get your plant back to a more natural shape. Another way to minimize shrub size is to cut one third of branches to the base each year, so there is always fresh new growth. Still, if a shrub is naturally too large for its location, choosing a better-sized replacement often works best.