- Orcish sword outside Whiterun, in a ditch being clutched by a skeleton hand.
- Elven dagger on the nord woman who runs the Riverwood inn. Have to pickpocket it.
- Dwarven dagger in the general goods store in Solitude. Must be stolen.
- Steel plate armour on the female merc you meet inside the Bannered Mare in Whiterun. Obtain it however you see fit.
- Scaled boots, helmets and gauntlets on the Stormcloak soldiers during the jagged crown quest. At least one will usually die.
- Scimitars on the shipwreck on the north coast.
- Dwarven arrows inside Solitude's fletcher. Must be stolen.
- Elven armour and weapons from Thalmor patrols. Can talk them into attacking you so they can be killed without getting a bounty.
- Dwarven weapons and armour inside the Dwemer museum in Markarth. Must be stolen.
- Staff of magelight next to the Jarls throne in Solitude. Must be stolen.
- Various enchanted jewelry worn by the Jarls. Must be pickpocketed.
Forsworn are - to my knowledge - the most powerful weapons that you can always find at level 1, apart from Skyforge steel.
I haven't visisted any Orcish strongholds yet, is it safe to assume you can always find Orcish gear there?