Doctors in our society tend to underestimate the dangers of medication and combinations of medication. If your doctor is concerned about you being on the pill or on any other medication, I would follow your doctor's recommendation and go off as soon as the doctor recommends. You may want to ...