There is nothing wrong in a man loving a woman and they intend to marry each other. Caste or creed or religion has noting to do with the desire to marry each other provided they both are eligible for marrying each other as per law of the land, i.e.,both should be single, both are of marriageable age. This criteria is more than enough for marrying each other.
When you people decided to marry each other you should not be worried about other consequences and should stand firm in your decision, you can take help of close and confident friends to make this happen.
Neither the police nor the parents of either side can stop the marriage legally, in fact you can seek police protection after or even before marriage.
Plan and execute the issue perfectly without any flaw.