Lets face it. Many of us have a lot of mods.
I am quite concerned with my mods. It might seem little for some, but 138 ESPs and ESMs total are quite a lot of mods being loaded in my skyrim. Not to mention that after clicking skyrim, i need to wait 30 sec...to load.
Plus i have about 10 more environment mods like skyrim HD.

How many more mods do i need before skyrim starts going berserk?

116. The game can load 254 ESMs and ESPs.

If you run into troubles earlier, it's not because of the amount per se, but because a few of them are bound to conflict with one another in interesting ways, if you're not careful.

*runs Morrowind with 172 mods*

255 active esp/esm files, and a limit of 400 active bsa/esp/esm files in the Data folder, I have read that Skyrim is very similiar to Oblivion, and the same rules do apply.
We may still be handicapped by the 4Gb memory cap for the game too, and texture replacers can hit this cap if the modder who made the textures is unaware of how much higher resolution textures choke graphics cards and RAM.