Why don't you answer the phone and let him speak. That might help you clarify your feelings about him. He surely would like to hear from you. If you are not at all interested then speak to him one last time and let him know your feelings. Its not fair to him to keep him dangling.