Put a recording device on your phone and record every call. Delete the ones that are not from him. From there bring these recordings to the police and get a restraining order. Even if he comes up to you and does nothing, he will still be breaking the law and either have to go to jail or court ...