Actually the amount paid by the buyer was earnest money (advance in token of sale) and you were suppoused to make a receipt for that with conditions that in case of default by the buyer the money gets forfieted. Even otherwise the money was earnest money and you were not bound to pay back as the deal was cancelled by the buyer.
Now since you have accepted the money as loan by signing the bond paper you have to pay
As the matter is in court you get some good lawyer who will plead for easy installments. You were free until signing of bond paper