I agree. I do believe the final decision is ultimately woman's and there can be no other way to approach this.The reason for not allowing the man to force an abortion onto a woman is that a fetus is considered to be a part of the woman's body, hence an abortion is a (pretty serious) interference with her physical integrity. I am not aware of any situation in which a decision regarding your body can be forced onto you in a horizontal relationship (short of parents-child, guardian-incapable etc.).
That being said, I do not believe that a decision made by one person, where the second person has no say whatsoever, should be binding and lifechanging for that second person.
When a woman says "I don't want to burden myself with a child" we, as pro-choice guys, want to give her an option to bail out. No matter what her reasons are. She doesn't want it? She can get rid of it, no questions asked.
We don't want the patriarchy to go "No, you're having that child and you're raising it. And you have no say in it."
The same should be true for a man. You don't want it? No one should be able to force you to become a father. At least legally. Otherwise its just weaponizing tyrannical patriarchy to go after men.