keep in constant touch with him wherever i go but yet he feels that i am doing something wrong. I have been doing every little thing to keep him out of doubt.
You ARE doing things wrong. You secretly go out to "have coffee"...of course this raises doubts in his mind.
Duh. Especially since you have a known history of being unfaithful.