Hi, 

These look like cabbage looper caterpillars which are very common on brussel sprouts. You can keep up the spraying and hand pick off any caterpillars you see. it may take some time to recover. Also its been very hot so double check the label on the neem to make sure you are using it in the temperature range. It could burn the leaves. 

A row cover will not help now since they are already there, but in the future it is a good idea to use one to prevent it before they come. 

Let us know if you have other questions beyond the management tips on the link. 

Emily