What can be done about this?

Keep a Watchman with Register  to  record details of visitors  & stop strangers from  entering your premises ;  put a sign board at the entrance that this is a private property and entry of strangers is strictly restricted and trespassers will be prosecuted. Complain to the police if anyone trespasses.

If your father has legal right to reside on the first floor of the house, you can not restrain his guests from visiting him unless you are harassed in anyway by the guests. In case of harassment you may contact the local police authorities for protection. 

May be, your father may have his own version. There is no scope for a remedy unless there is violation of any law on the part of your father or his 'guests'.