Unfortunately this plant won't recover, but it's hard to determine exactly what the culprit was. Skimmia is known for being sensitive to soil drainage, and is intolerant of its roots staying too wet. Therefore, our prime suspect would be a type of root rot, though they can be caused by several different fungi. There is no curative fungicide for this, and the few that work preventatively are not practical to use since they (a) don't always work even when applied as directed, and (b) don't address the core problem, which is the root stress that preceded infection by these opportunistic pathogens. (These fungicide formulations also tend to be more expensive than other formulations, and might have negative impacts on the beneficial soil fungi plants rely on to thrive.)

For now, all you can do is to make sure the other Skimmia aren't being over-watered, either through irrigation or via surplus moisture from a nearby roof downspout outlet or other drainage patterns that shunt runoff into their root zone. When monitoring for irrigation needs when rains are insufficient (and especially when they are competing for moisture with nearby tree roots, if they are in tree shade), feel the soil and only water when it's becoming relatively dry to the touch at a depth of about six inches.

Ironically, yes, under-watering is another possibility as its symptoms are nearly identical to over-watering, though less likely in this case since Skimmia tends to tolerate droughty conditions better than they do sogginess. A few fungal pathogens take advantage of drought-stressed plants and cause stem-killing infections, though it may be less likely they'd cause all branches to die at the same time as appears to have happened here.

We can't see the crown (where stems emerge from the roots) in the photo, but if mulch is placed right up against it, move it aside a few inches in all directions to allow for air circulation to discourage disease, since some plant infections can begin at the soil line in the crown, cutting-off moisture and nutrient flow between the roots and the foliage.

Although not a foolproof diagnostic tool, you can see if the roots appear to have died from "drowning" and rot by inspecting them after you remove the plant. Roots that have a mushy outer layer that easily separates with a gentle tug from their stringier inner core are suspected to have died from issues related to too much moisture. Roots that died from other causes generally tend to snap cleanly in two and take more force to break. Secondary decay of already-dead roots could complicate this assessment, though, so it's just a guideline.

There is no need to treat the soil with anything if you wish to replant; these are ubiquitous fungi and could recolonize treated soil at any time when conditions are suitable, so eradication isn't feasible. However, since they are largely diseases of opportunity, a healthy plant well-matched to the site conditions will not necessarily have the same problem.

Miri

Unfortunately, yellowing and drooping leaves is not a condition that Skimmia (or any other evergreen, really) recovers from, especially when this extensive to the entire plant. It suggests there has been drastic root loss, though whether from overly-wet / poorly-drained soil, under-watering, or other issues is hard to tell since symptoms can overlap once the roots decline. If the plant can be pulled out of the ground and appears to have roots in the same shape as the original container, then it didn't establish, though that issue can still overlap with others, like infection.

Skimmia is not often grown in our area due to being relatively picky about site conditions (even though garden centers still stock it on occasion). We see them fail regularly (even in well-drained nursery pots) from presumed root rot, though without a lab culture of the roots, we can't determine if it was due to Phytophthora, Pythium, Thielaviopsis, or one of several similar, opportunistic soil pathogens. You could investigate the soil acidity (pH) in that area via a laboratory soil test, in case it's too alkaline for the roots to thrive. Lab soil tests will be more accurate and detailed than home test kits. For root rot, sometimes a pH adjustment to more acidic levels will help suppress the fungus to the point that roots can resist invasion. These fungi are ubiquitous in the environment, so eradication via fungicide is not possible or practical to attempt. References state that Skimmia prefers acidic soil, though they don't all agree, since some state it can take neutral soil as well. Even so, since it can handle acidic soils, that is probably a good goal since it can deter the fungi above.

There is no cure for plants declining from root rot, so affected plants need to be removed and disposed of (don't compost). You can try to confirm that might have been a factor by inspecting the roots after removal. If they are browning or dark, and especially if they are mushy and pull apart easily, that is suggestive of infection. Those roots will have a mushy outer layer that pulls away with little effort, leaving behind a stringy or hair-like inner core. In comparison, live roots should be paler and firm, and roots dead from other causes tend to require more force to break, and they will usually snap cleanly in two instead of separating into layers.

You don't need to treat the soil per se, though improving drainage might be useful if you wanted to grow species in that area that are sensitive to wet conditions. Or, try to select a replacement that is less vulnerable and which can thrive in more evenly-moist soils if you suspect they stayed too wet. In semi-shade with acid soil, for example, that might mean that Leucothoe or a dwarf Illicium would do better (the latter being a little less cold-hardy than the former), or in near-neutral soil, a conifer like Japanese Plum Yew (Cephalotaxus). If you prefer something short and evergreen with red berries, maybe a miniature holly like variety 'Rock Garden' would fare better, though it would be even more slow-growing than the Skimmia and, being somewhat rarer, more expensive. Osmanthus variety 'Kaori Hime' (a.k.a. 'Party Princess') has tiny holly-shaped leaves and fragrant small white flowers in autumn, and stays compact compared to other varieties of Osmanthus. Sweetbox (Sarcococca) comes in both ground-covering (S. humilis) and shrubby forms (S. confusa and S. ruscifolia) that usually does well in semi-shade or shade. If they produce berries at all, they're not very decorative, but their small flowers open very early in the year and are quite fragrant.

Miri