That's strange. It might be a newer bug related with the 2.55 release. I'm pretty sure that if you switched back to Khergit in 2.53 and 2.54, your troops would revert into Khergit. I haven't really played with 2.55 enough to know if it's not the case anymore.
Just to check, troops won't instantly change cultures just because you said so. When you change cultures, your new recruits should come from the culture you set from that point on, including new patrols and troops recruited by the lords in your kingdom. Existing patrols, troops already in the garrison, etc., won't suddenly become Khergit if they're Swadian. If your garrison is already full then you won't get any new recruits until you empty it somehow.
The placeholder thing I was talking about was referring to "Player Faction" troops. They have Swadian troops as a template since Swadia is fac_kingdom_1 in the game files. If you picked Player Faction without making your own Player Faction troops via editing the mod, they'll show up as Swadian.