Thanks for the feedback. I'll have to make my mind up considering the castelans, while they would benefit the army in-game, the current list just completely has this 'raiding theme', something which units who are supposed to be the defenders of Umbar wouldn't really fit in to.
As for the low defense, I indeed added a full warband of arbalesters to win any shootouts. All bow units apart from other crossbowmen need 3 hits for every one hit I need against most archers (defense 3/4) so apart from maybe a Grey Company or Uruk force, I should probably be able to discourage any opponent to try and skirmish against me.
Seems like this army might indeed be good enough as it stands. That means at least one finished option for my choice of a new army
.