Just send emissary first to declare war if you don't want to engage in any battle but to directly attack a specific town.
Edit: Sorry I guess you just released from vassalship without taking any of your previous Ruler fief, so in this case you are not a ruler yet. In that case yes you need to attack their troops. A quick tips from me. Just be calm, and do trading, when the war break up, side up with one of this faction, and take a city that was been taken so the garrison is lower than normal and the time you require to complete the siege also shorter.