As a Principal Employer, the Company has responsibility and liability.
If you have already paid the amount to the contractor, then ask the contractor to produce proof that he has made payments to the workers. Otherwise, stop further payments to him.
In fact you are liable to ensure that he properly remits the EPF amounts etc. with the authorities.