Shorting the connection to start the computer is a common practice. I personally don't do it, but as long as you don't hold it there and remove it once you see the fans kick in then you wont damage anything.

What do you do to start up the pc?

I'm sorta mixed lazy and impatient and just put it all into the case so I am able to use the power button. Ive been working with computers for over 10 years so I skip a lot of the in-between steps.

He also said that you can accidentily discharge electricity to the components while your working on them. Could you prevent this by wearing rubber gloves?

Yes, but an esier thing to do is either get a Anti-static wrist strap, or even easier, touch the power supply every so often. Basically keep in contact with the powersupply or case frame to stay grounded.

Is Newegg a good place to get components or should I consider another place to get parts from?

It is indeed a good place. Though shopping around couldnt hurt.