green algae growing on my aluminum siding and wooden fence
Having algae or mossy growth on the north or shaded-sides of buildings is not uncommon, and given the incredible rain we had last season, it's not surprising.
The easiest thing to use is a power washer at the gentlest setting useful so you don't remove color or wood grain.
There are commercial products available at hardware stores and garden centers too but read and follow label instructions carefully.
Scrubbing with a weak10% bleach solution could also work, but protect sensitive plants from run-off.
Just to be sure on the fence... you are not talking about lichens are you?
Here is a page on what they are: https://extension.umd.edu/hgic/topics/lichens
They are harmless and don't require removal.