Grounds for Nullity Proceedings in a Hindu Marriage
1.If a marriage has not been consummated due to impotence of either party.
2.If the bride was pregnant at the time of the marriage by some person other than the bridegroom and the latter was not aware of it at the time of the marriage. If the bridegroom knew of this before the marriage, the marriage cannot be nullified.
3.If the consent of one of the parties to the marriage was obtained by fraud or force.
Impotence should not be confused with ability to procreate. Due to vericocele probably your semen may not have adequate quantity of moving cells, and consequently the chances of impregnating may be less. But that cannot be called 'impotence'. Impotence according to my understanding is more to do with 'erection' and 'penetration'. You have to take appropriate medical opinion to rebut the allegation. Give medical certificate about his impotency in a court.
File a divorce petition on the ground of impotency but it will take some time to get order from the court.
For further readings go to http://www.indidivorce.com/