I have played Fallout 3 a lot, but I am confused as I start New Vegas. Are all of the books and healing potions temporary? What is the best way to level up and what is a good starting weapon?

There are skill magazines and skill books.
Magazines give +10 temporary skill boost (With comprehension perk you get +20)
And books give +3 permanent increase (With comprehension perk you get +4)

The best way to level up is to do quests, kill stuff, lockpick, hack etc etc.

Good starting weapon?
Depends on your combat skill.
Energy weapons have Incinerator which you get off of the convict leader in Primm.
Melee have Chance's knife which is a bit tricky to get to. http://fallout.wikia.com/wiki/Chance%27s_grave (Need a shovel, buy one from Chet in Goodsprings)
Guns have everything.
Unarmed has that claw gauntlet which you can buy from Chet in Goodsprings, Nash in Primm and various merchants at Mojave Outpost.
And Explosives have squat until you get the Grenade Machine Gun.