I say as soon as they are ready. I don't feel that parents should unnecessarily overwhelm their children with advanced academic situations, but we also shouldn't underestimate their intelligence. If a child is showing signs of readiness, then, by all means, teach them whatever they will absorb!
--Free Kids Books --
Free is right. If a child shows interest even before they're in school, foster that interest by introducing them to numbers. Let them get comfortable with their learning and they'll turn out to be life-long learners.
Math: Never too early to help children learn about numbers one way or another, or rather to experience numbers in age-appropriate ways. Imagine a long trip with a child. It's going to last three hours. After 10 minutes, you hear: "Are we there yet?" You can answer: "If this trip was a dollar, we've only traveled 5 cents." There is no need to be precise; answering 1 cent would be fine. Then, the estimate is updated: "25 cents....50 cent...We're just a few cents away from a dollar." At the end of the trip, the child is given the dollar. The child is taught about money, about percents in an indirect way, about feeling good about numbers.
Your baby starts to learn as soon as he is born. From birth to 1 year of age, a baby is able to look at pictures in books and is calmed by rhyming stories. At age 1 to 3 years, an infant can respond to the stories you read as well as the sound of your voice. During this period, they often pretend to read like their parents. At this age level you can start the journey of teaching your child about math, reading and writing.
For teaching math, the best time is after they birth. Glenn Doman has mentioned the development of baby's brain and how to teach baby math by using a right method. For more resource: http://babycan.org/
Other people asked questions on similar topics, check out the answers they received:
Other people asked questions on various topics, and are still waiting for answer. Would be great if you can take a sec and answer them