Proper cav 1v1s take a long time. If it doesn't take long it's because one or both of the players are playing (very) suboptimally. Incidentally, it's a large part of why cav duels between good players aren't that interesting to watch.
The rules as written basically state "you're not allowed to play your best" which isn't very interesting either.