It's either real men Drink Mead (And get their calcium from Cheese), or they're calling you a baby - some people use the phrase in the context that makes it clear they think you're an infant.
The nitpicker in me insists that I must point out that one can get all the calcium required from a normal diet that includes no dairy at all.
In fact, the enzym that allows advlts to digest dairy is a relatively new feature, evolutionary speaking. It has not spread across the entire populace yet.
Many Asians and Africans are lactose intolerant and the habit of consuming dairy products is really only prevalent among western cultures.
The idea that milk is needed or good for you probably comes from decades of commercials payed for by the dairy industry to promote the use of their products. These are often given an 'official' shade by getting endorsemants from various government agencies that have very much to gain themselves from selling more milk.
Some people claim that dairy is in fact bad for you when you are no longer an infant, and that idea has some merits. Lactose is still a foreign substance to the body, and dairy stimulates the production of mucus.
Some people who have completely quit dairy claim it gives them better overall health, skin tone, healthy hair and less brittle nails.
I have not tried this myself as I cannot live without cheese (I never met a cheese I didnt like.)
Of course, in moderation it remains a good source of certain vitamins and minerals.
Still, the moral I suppose is that you really shouldnt believe everything you are told by those in authority.
*Takes off nitpickers hat*